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Authors' conclusions 


Farmer field schools (FFS) are a common approach used to transfer specialist knowledge, 
promote skills and empower farmers around the world. At least 10 million farmers in 90 
countries have attended such schools. 

FFS are implemented by facilitators using partidpatoiy "discovery- based" learning based on 
adult education principles. Many different implementing bodies have been involved. Field 
schools have a range of objectives, including tackling overuse of pesticides and other harmful 
practices, improving agricultural and environmental outcomes, and empowering 
disadvantaged farmers such as women. 

We conducted a systematic review of evidence on FFS implementation to investigate whether 
FFS make a difference, to which farmers, and why or why not. We synthesised quantitative 
evidence on intervention effects using statistical meta- analysis, and qualitative evidence on 
the barriers and enablers of effectiveness using a theory of change framework. 

The results of statistical meta- analysis provide evidence that FFS are beneficial in improving 
intermediate outcomes relating to knowledge learned and adoption of beneficial practices, as 
well as final outcomes relating to agricultural production and farmers' incomes. The findings 
suggest this to be the case for FFS promoting integrated pest management (IPM) technology, 
as well as other techniques. However, the rigorous impact evaluation evidence base is small 
and there are no studies that we were able to identify as having a low risk of bias. 

There is no evidence that neighbouring non- participant farmers benefit from diffusion of 
IPM knowledge from FFS participants. Therefore, they do not experience improvements in 
IPM adoption and agriculture outcomes. 

The evidence of positive effects on agricultural outcomes is largely limited to short-term 
evaluations of pilot programmes. In the few examples where FFS have been scaled up, the 
evidence does not suggest they have been effective in improving agricultural outcomes among 
participating farmers or neighbouring non- participants. 

Although empowerment is a major objective of many FFS, very few studies have collected 
information on this outcome in a rigorous manner. A few studies suggest farmers feel greater 
self-confidence. 

What explains the lack of scalable effects among FFS participants, or diffusion of IPM 
practices among the community? FFS differ from standard agricultural extension 
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interventions, which tend to focus on disseminating knowledge of more simple practices such 
as application of fertiliser and pesticides, or adoption of improved seeds. The experiential 
nature of the training, and the need for the benefits of the FFS technology to be observed, are 
barriers to spontaneous diffusion. Furthermore, the effectiveness of scaled- up interventions 
has been hampered by problems in recruiting and training appropriate facilitators at scale. 

The review provides implications for policy, practice and research. 
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Executive Summary 


BACKGROUND 


After almost three decades of decline in public support, agriculture is now back on the 
development agenda. Since the late 1980s, support to agriculture has shifted from top-down 
approaches to those identifying technologies and methods of communicating technologies 
which are suitable to support farmers' livelihoods in a sustainable manner, including 
participatory approaches based on the notion of creating spaces for farmer self- learning. One 
such approach is the farmer field school (FFS), an adult education intervention which uses 
intensive "discovery- based" learning methods with the obj ectives of providing skills in such 
areas as integrated pest management (IPM) and empowering farmers and communities. FFS 
have been implemented in 90 countries worldwide, reaching an estimated 10- 15 million 
farmers. Farmer field schools may appear to be the latest tool, but what does the evidence 
say regarding their effectiveness? 


OBJECTIVES 


This systematic review synthesises evidence on interventions identified as "farmer field 
schools" conducted in low- and middle- income countries. The review aims to provide 
answers to the following research questions: 

Review question ( 1) : 

a ) What are the effects of farmer field schools on final outcomes such as yields, net 
revenues and farmer empowerment? 

b) What are the effects of farmer field schools on intermediate outcomes such as 
knowledge and adoption of improved practices (e.g. reduced use of pesticides)? 

c) What are the effects on outcomes for non- participating neighbouring farmers living 
in the same communities as FFS farmers? 

Review question (2): What are the enablers of and barriers to FFS effectiveness, diffusion 
and sustainability? 
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STUDY SELECTION CRITERIA 


Studies induded in the review satisfied the following criteria. 

Eligible partidpants induded fanners growing arable crops, living in low- or middle- income 
countries at the time of the intervention. The review included those partidpating directly in 
the field school and also non-partidpant neighbour farmers who may benefit through 
spillover effects or more formal dissemination methods. 

Eligible interventions were those identified as "farmer field schools," regardless of the design 
or implementation, induding FFS programmes providing training in I PM and other 
techniques. Studies combining FFS with other intervention components, such as input or 
marketing support, were also induded. 

Comparisons eligible for the effectiveness review were farmers who received no intervention, 
or access to agricultural extension services from another source, including IPM (or 
equivalent) training. 

All outcomes reported were eligible for the review, primary Primary outcomes were 
agricultural outcomes , induding yields and profits (net revenues). Secondary outcomes 
included other final outcomes such as environmental outcomes, health status and 
empowerment; and intermediate outcomes, including fanner knowledge and adoption of 
practices. Qualitative evidence on barriers to and enablers of effectiveness and sustainability 
were also induded, induding process and implementation information and measures of 
benefidaries' attitudes and experiences with FFS. 

Eligible study designs for the effectiveness synthesis (review question 1) were measurable 
using counterfactual impact evaluations, induding experimental or quasi- experimental 
study designs and methods of analysis. Studies eligible for the synthesis of barriers and 
enablers (review question 2) were based on primary data colleded from FFS partidpants, 
extension agents or experts, analysed using qualitative methods or descriptive statistics. The 
qualitative studies needed to report at least some information on the research question, 
procedures for collecting data, sampling and recruitment, and at least two sample 
characteristics. 


SEARCH STRATEGY 


The search included electronic academic databases, internet search engines, websites and 
theses, as wdl as handsearches of key journals and literature snowballing. Searches induded 
general sodal sdence sources as well as agriculture subjed-spedfic sources of published and 
unpublished literature. All searches were updated in October 2012. 
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The farmer field schools evaluation community has generated a large number of evaluations. 
We screened the titles and abstracts of over 28,000 papers, the majority of which were 
irrelevant to the topic. Four- hundred- sixty (460) relevant papers on FFS were assessed for 
inclusion based on full text. After the final screen by two authors, 134 quasi- experimental 
studies comprising 92 distinct evaluations meeting the inclusion criteria were eligible for the 
review. The impact evaluations provide quantitative estimates of effects on outcomes for 71 
FFS projects. However, only 15 of the impact evaluations meeting the inclusion criteria were 
judged to be of sufficient internal validity to make predictions for policy. The review also 
includes 20 qualitative evaluations meeting the inclusion criteria, which discuss the barriers 
to and enablers of change in 20 FFS projects. A portfolio review of 337 project documents 
was also conducted. 


DATA COLLECTION AND ANALYSIS 


Two independent reviewers assessed the full text papers against the inclusion criteria; 
discrepancies were resolved by consensus or by a third author if needed. Two reviewers 
extracted data from included studies. Quantitative impact evaluation studies were critically 
appraised according to the likely risk of bias according to threats to internal validity (causal 
identification), external validity (generalisability) and file-drawer effects (publication bias). 
Qualitative evaluations were assessed according to adequacy of reporting, data collection, 
presentation, analysis and conclusions drawn. 

We used a hypothesised programme theory of change (White, 2009) as the framework for 
integrating the evidence. We collected data on programme design, implementation, targeting 
and contextual factors, and finked individual studies by programme in order to assess 
whether heterogeneous programme effects were correlated with study design, 
implementation and context. 

For the quantitative synthesis (review question 1), we extracted effect size estimates from 
included studies, calculating standard errors and 95 per cent confidence intervals using data 
provided in the studies, where possible. We used random effects meta- analysis, estimating 
average effects of farmer field schools on the different outcomes, and examining 
heterogeneity. The results of the publication bias analysis suggested under-reporting of small 
sample studies with negative or insignificant findings for studies reporting evidence on 
agricultural yields, which is evidence for possible publication bias. 

For the synthesis of qualitative evidence (review question 2), we used a thematic approach 
(Thomas & Harden, 2008), combining predetermined themes based on the finks and 
assumptions in the theory of change model, as well as any other themes emerging from the 
detailed coding of the included studies. 

In the final stage of analysis, we used an iterated approach in which some effect moderators 
identified during the qualitative synthesis were tested in meta- analysis and meta- regression. 
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RESULTS 


Review question (1) 

No studies with a low risk of bias were identified for the review of effects and only 15 ( out of 
92) quasi- experimental studies were assessed as being of medium risk of bias and therefore 
policy- actionable. The results of these medium- risk- of- bias studies (reported in Summary of 
Findings Table 1) suggest farmer field schools impact positively on intermediate and final 
outcomes for participating farmers in the short to medium term. 

Findings for intermediate outcomes were as follows: 

• There was a significant increase of 0 . 2 1 standard deviations on knowledge about 
beneficial practices among farmer field school participants over comparison farmers 
(SMD=0.21, 95% confidence interval (Cl) =0.07, 0.35; Q=5, Tau-sq=0.008, 1- 
sq=55%; evidence from 3 studies). 

• There was a significant reduction in pesticide use by 23 per cent for IPM and IPPM 
FFS participants over comparison farmers (RR=0.77, 95% Cl =0.61, 0.97; Q=40, Tau- 
sq=0.07, 1-sq=83%; 8 studies). Effects on pesticide use were particularly large and 
consistent for cotton IPM projects in Asia. 

• There was a significant increase in indices of adoption of other beneficial practices 
by 0.22 standard deviations over comparison farmers (SMD=0.22, 95% Q =0.06, 
0.38; Q=10, Tau-sq=0.02, 1-sq=80%; 3 studies). 

For final outcomes, the findings were as follows: 

• A significant increase in agricultural yields was estimated among FFS participants, 
by 13 per cent over comparison farmers (RR=1.13, 95% Q =1.04, 1.22; Q=53, Tau- 
sq=0.008, 1-sq=81%; 11 studies). 

• A significant increase in profits (net revenues) was estimated, by 19 per cent among 
FFS participants over comparison farmers (RR=1.19, 95% Q =1.11, 127; Q=l, Tau- 
sq=0, l-sq=0%; 2 studies) . The increase in profits was higher for FFS projects which 
also included complementary interventions involving input or marketing support 
(RR=2.51, 95%a=1.51, 4.16, Q=l, Tau-sq=0, l-sq=0%; 2 studies). 

• There was a 39 per cent reduction in estimated environmental impact quotient (EIQ) 
score as a result of reduced pesticide use among FFS farmers over comparison 
farmers (RR=0.61, 95% Cl =0.48, 0.78; Q=3, Tau-sq=0.01, 1-sq=33%; 3 studies). 

• We could not identify any studies which provided valid estimates of impacts on 
farmer health outcomes. 
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• Very few studies assessed empowerment using quantitative counterfactual methods, 
and only one provided estimates of statistical precision. 

However, there is no evidence of effects on outcomes over the longer term (follow-up surveys 
greater than two years after implementation) in programmes which have been scaled up 
nationwide. 

For IPM farmer field schools, there is no evidence that diffusion from FFS participants to 
non- participating neighbour fanners usually happens: 

• Overall, studies found no significant change in knowledge among FFS neighbours 
over comparison farmers. There was also no evidence for improvements among 
neighbours on pesticide use, yields or environmental impact quotient. 

• When relatively better- educated farmers are targeted to participate in the I PM field 
schools, diffusion may occur for simple practices (such as reduced pesticide use) and 
yields. However, even in a few cases where diffusion appeared to occur, the evidence 
does not suggest diffusion to non- participants is sustained over time. 

Review question (2) 

Qualitative evaluations (reported in Summary of Findings Table 2) in the review helped us to 
understand the different types of farmer field schools implemented around the world, the 
reasons for heterogeneous impacts among FFS participants, and the limited diffusion to 
non- participating neighbour fanners. FFS use discovery- based learning methods which 
differ from agricultural extension interventions that tend to focus on disseminating 
knowledge of more simple practices, for instance application of fertiliser and pesticides, or 
adoption of improved seeds. However, there are several barriers to spontaneous diffusion of 
knowledge and practices. The FFS curriculum is complex and the training should be 
experience- based, so that farmers are able to observe that FFS practices have a relative 
advantage over conventional farmer practices. Existing levels of social capital, the reach of 
social networks, and approaches to targeting FFS participants were found to be potentially 
important factors in influencing diffusion. More generally, the studies identify some of the 
more common problems in implementation, notably where a top-down “transfer of 
technology" approach has been implemented for an intervention which is intended to be 
based on a "bottom- up" participatory approach. All qualitative evaluations presented some 
evidence of use of triangulation to verify their findings, although most studies had 
weaknesses in reporting on sampling, analysis, and presentation of data, making quality 
appraisal of this evidence base challenging. 


IMPLICATIONS FOR POLICY AND PROGRAMMES 


Fanner field schools can have beneficial effects for participating farmers, in pilot 
programmes in the short term. The impacts on agricultural outcomes may be of substantial 
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importance to farmers, in the region of a 10 per cent increase in yields and 20 per cent 
increase in profits (net revenues) . The effects are particularly large when FFS are 
implemented alongside complementary upstream or downstream interventions (access to 
seeds and other inputs, assistance in marketing produce) for cash crops. 

However, the few studies of scaled- up programmes measuring outcomes over the longer 
term (more than two years post- training) do not find any evidence of effects of FFS. Farmers 
may also feel more confident, but again very few studies have assessed empowerment 
outcomes rigorously. 

There is little evidence of diffusion of improved practices or outcomes from FFS participants 
to non- participating neighbour farmers. Field schools targeting more educated farmers may 
be better able to diffuse simple practices, such as on reduced pesticide use, than field schools 
that target less educated farmers. However, there is no evidence that any diffusion of 
practices is sustained over time, nor any evidence for adoption of more complex IPM 
practices via diffusion. 

As a method of rural adult education, FFS appear suited for gradual scale-up provided there 
is a dear focus on ensuring local institutionalisation (i.e. favouring intensiveness of coverage 
in each community over geographical breadth of coverage) . On the other hand, FFS seem 
unsuited to solve the problems of large-scale extension. The approach may not be cost- 
effective compared with agricultural extension in many contexts, except where existing 
farming practices are particularly damaging, for example due to overuse of pestiddes. This is 
because of the highly intensive (and therefore relatively costly) nature of the training 
programme, the relative successes in targeting more educated farmers as compared with 
disadvantaged groups, and failures in promoting diffusion of IPM practices. 

Targeting FFS partidpants: Proponents of FFS have recommended targeting more highly 
educated farmers, those with greater land endowments, younger farmers and women, 
favouring those with relatively low opportunity costs of labour or farmers with relativdy high 
pestidde costs. Problems were highlighted in targeting women who lived in household where 
they were not in a dedsion- making position, and youth who were unable to dedicate 
suffident time to the FFS plot or their fields. 

Where the aim is to indude women and disadvantaged members of the community, 
implementers may need to tailor the intervention to enable their partidpation in the 
programme. The curriculum needs to be relevant and consistent with the needs and 
opportunities of women and the poor. Most obviously, in contexts where women are 
primarily responsible for growing subsistence crops, a curriculum that covers only 
commercial crops is unlikely to attrad women partidpants. More generally, the curriculum 
and crops covered in FFS should also be adapted according to the local agricultural system 
and the needs of the farmers targeted by the programme. Curricula need to deal with the 
major challenges fadng farmers. In most cases, these challenges will be multifaceted, 
highlighting the need to balance comprehensiveness with being able to cover all issues in 
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sufficient depth to ensure appropriate learning. A cumulative approach over several seasons, 
including exchanges between field schools, may be preferable. 

FFS facilitators: The evidence also suggests that appropriate targeting and training of FFS 
facilitators is important. The theory of change suggests FFS should be delivered according to 
a participatory and discovery- based approach to learning, including opportunities for 
farmers to experiment and observe new practices, particularly if farmers are to be 
empowered with lifelong skills capacity development. Attempts to target facilitators based on 
education or literacy levels may be less effective than targeting based on ability to 
communicate, and appropriate training which enables facilitators to use a bottom- up 
approach. This is most obviously a barrier in scaled- up programmes where FFS facilitators 
are recruited from extension staff who previously used more top-down agricultural extension 
methods. Recruitment of facilitators should take into account personal attitude, maturity, 
literacy, leadership skills, knowledge in local language and experience with farming. In many 
contexts the gender of the facilitator should be carefully considered. Facilitators should have 
access to ongoing support and backstopping from supervisors and technical experts 
connected to local research centres. Regular monitoring of facilitators may help to identify 
schools where additional support is required. 

Complementary policies: Institutional actors involved in FFS should consider fanners' 
needs and interests in the design and implementation of the FFS programme. In some 
contexts stronger policies and regulatory measures may be necessary to counteract the 
activities of the pesticide industry, including the promotion and sale of pesticides by 
extension workers who are promoting FFS. New policies facilitating participatory 
agricultural extension approaches, replacing earlier extension policies aimed at promoting 
off-the-shelf technologies and input packages, may also be necessary. 

Local institutionalisation: Formal support and encouragement of FFS alumni, including 
technical assistance and backstopping, may be important for the sustainability of FFS 
practices and related activities. Given the skills- based nature of the practices promoted in 
FFS, formal community- building activities, support and successful attempts to 
institutionalise the approach, to encourage FFS graduates to train other farmers, are likely to 
be needed for any broader diffusion to non- participating neighbour fanners, although the 
evidence base does not indicate that such attempts have been successful in the past. 


IMPLICATIONS FOR RESEARCH 


The majority of FFS impact evaluations (68 out of 92) use designs of questionable internal 
validity, and are therefore of limited value in determining whether farmer field schools have 
made a difference to outcomes. We were not able to locate any completed evaluations which 
used randomised assignment, an approach which is feasible for FFS. In three-quarters of 
evaluations, no serious attempts were made to control for confounding through statistical 
matching or other statistical analysis, and in one- third of cases statistical significance tests 
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were not reported. The likely consequence, as indicated in the meta- analysis, was the 
systematic overestimation of effects for all outcomes. The extent of resources that has been 
devoted to farmer field schools evaluations might therefore be usefully re- allocated to 
conducting fewer but more rigorous impact evaluations, particularly those based on a solid 
counterfactual, with prospective duster- level assignment (randomised or otherwise) to allow 
measurement of community- wide diffusion and to assess effects on agriculture and 
empowerment outcomes in the medium to longer term (three years or more) . 

Evaluations should report information on both intervention design and implementation 
processes so that it is possible to assess whether programme causal chains break down 
because the intervention design is simply not appropriate for the context or because of poor 
implementation. 

Many qualitative evaluations need to report aspects of the research process in greater detail 
to allow users to assess their credibility and applicability. In particular, clear reporting on 
objectives, on methods of sampling, data collection and analysis should be provided. Greater 
use of structured abstracts will fadlitate easier access to quantitative and espedally 
qualitative research. Future studies should include data on views and experiences of FFS 
facilitators and agricultural extension workers. 
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Summary of Findings Tables 


Summary of Findings Table 1: Effectiveness studies (review question 1) 


Outcomes 

Summary of findings 

Relative 

No. of effect P S?? Quality Statement 

studies size ... assessment 3 

(participants) ,95% 

Final outcomes - all 

farmer field school participants (review question la) 

Yields (primary 
outcome) 

13% increase ++oo 

in yields of Low 

, . , FFS FFS may increase yields of FFS 

,, ' ! participants on Moderate participants by an average of 13% 

(3 198) (1 04 average risk of bias relative to comparison group, though 

’ ^ 22 ) relative to and there is notable variation across 

comparison publication populations and contexts 

group bias strongly 

(4%, 22%) suspected 

Net revenues 
(primary outcome) 

19% increase 

, ++00 

in net revenue , 

ofFFS Low 

1.19 nartirinantc nn F FS may increase net revenues 

2 RR p p Moderate (profits) of FFS participants by an 

(488) (1.11, iff risk of bias average of 19% relative to 

1.27) /nmJJricnn and small comparison group 

number of 

(11%, 27%) studies 

Empowerment 

FFS +ooo 

participants Very low 

1.13 more 

2 13 likely to report Moderate 

1 ^ positive risk of bias, The evidence on the impact of FFS on 

(2001 ,, empowerment serious empowerment for FFS participants is 

, outcomes indirectness inconclusive 

1 ' relative to and very 

comparison serious 

group imprecision 

(0.46,2.12) 

Environmental 
outcomes 
(environmental 
impact quotient) 

39% reduction 
in 

environmental ++oo 

impact Low 

3 RR 1 ,u r°' Moderate FFS may reduce the environmental 

(1,149) (0.48, participants® rtskofbias 'mpa ctquo ent by 39% on average 

0.77) average and small relative to comparison group 

relative to number of 

comparison studies 

group 

(52%, 23%) 
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Outcomes 

Summary of findings 

No. of Relative Percentage Quality statement 

studies effect change assessment 3 

(participants) size compared with 

Intermediate outcomes - farmer field school participants (review question lb) 

Knowledge test 
scores 

The 

knowledge 
test scores 
achieved by 

FFS t 

participants , 

n , 1£ . Mn2 areon ° W FFS may increase knowledge of FFS 

3 ' (0.07, a ; er Jf Moderate risk P artici P ants ^ °' 2 |. standard deviations 

(426) 0.35) * 2 } a of bias and on average relative to comparison 

' ta ; dard small number group 

greaterthan ofsUies 
in the 

comparison 

group 

(0.07,0.35) 

Pesticide use 
(IP M/IP P M FFS 
only) 

17% 

decrease in 

.. ■ i ++00 

P . - r r Low F F S may decrease pesticide use of 

g 0.83 RR usaD f ^ IPM/IPPM FFS participants by 17% on 

n (0.66, P ar lc| P an s Moderate risk average relative to comparison group 

1.04) relative to of bias and though there is notable variation across 

reaiv .° serious populations and contexts 

comparison ... r r 

imprecision 
group r 

(-34%, 4%) 

Adoption of 
beneficial practices 

The number 
of practices 

adopted by +ooo 

FFS Very low 

participants 

, 0.22 SMD lson Moderate risk Evidence on the effect of FFS on the 

hc \ a \ (0.06, v n ^ 0 . ias ' adoption of beneficial practices is 

l794 ’ «' 38 » standard incoSncy 

deviations and small 

greaterthan number of 

in the studies 

comparison 
group 

Diffusion to neighbour farmers (review question lc) 

Pesticide demand 
neighbours 
(pesticide use, 
pesticide costs) 

No statistically 
significant -F+oo 

og5 effect on Low 

8 ," R , peS “ F e s USe Moderate FFS may not have any diffusion effect 

IU151 ,0 3 64 ; neighbours nskofbias on pesbcide use 

relative to and serious 

comparison imprecision 

group 
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Outcomes 

Summary of findings 

No. of Relative Percentage Quality statement 

studies effect change assessment 3 

(Darticioants) size comoared with 

Yields 

No statistically 

. . e ’ +400 

significant . 

1.02 effect on the L0W 

4 RR yields of FFS M . , FFS may not have any diffusion effect 

19861 * r iB 

compaj" inc “"“ s ncy 


Notes: 1/RR = response ratio. 2/SMD = standardised mean difference. 

3/ The rating guide used for the assessment of the quality of the evidence was adapted from GRADE and is available from 
the authors. 

Source: authors based on GRADE. 
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Summary of Findings Table 2: Barriers to and enablers of effects (review question 2) 


Outcomes 

No. of 
studies 

Statement 

Barriers to and 
enablers of 
knowledge 
acquisition 

17 

studies 

Barriers: FFS facilitators do not receive sufficient training and ongoing support (6 
studies). Facilitators do not have enough farming experience and/or appropriate 
characteristics (2 studies). Lack of adequate and timely resources for FFS schools (3 
studies). Farmers excluded due to restrictive targeting criteria or procedures (4 studies). 
Farmers unable to participate due to gender, cultural norms or poverty (7 studies). High 
levels of drop-out due to incorrect expectations or lack of interest, access or time (7 
studies).T raining delivered in a top-down manner, using transfer of technology approach 
(4 studies). Curriculum not appropriate or relevant to the local context (7 studies). 
Imbalance in relationship between farmers and facilitators (3 studies). Facilitators use 
national language, in which farmers are not fluent, or too many foreign and scientific 
terms (2 studies). 



Enablers: FFS facilitators have experience with farming, are literate and mature, and 
have a positive personal attitude and leadership skills (3 studies). Gender of facilitator 
acceptable to participants and their families (2 studies). Farmers motivated to learn and 
improve livelihoods (5 studies). Training delivered in a participatory, bottom-up manner 
(9). Curriculum appropriate and relevant to the local context (7 studies). Facilitators use 
local language and concepts and metaphors common to farmers (2 studies). 

Barriers to and 
enablers of adoption 
of FFS practices 

18 

studies 

Barriers: Training delivered in a top-down manner, using transfer of technology 
approach (4 studies). Curriculum is not appropriate and relevant to the local context (7 
studies). Farmers do not observe benefits from FFS practices (2 studies). Practices too 
complex for farmers to implement (3 studies). Farmers lack access to inputs, capital 
and/or markets (5 studies). Low levels of social capital among participants (1 study). 

Enablers: Training delivered in a participatory, bottom-up manner (9 studies). 
Curriculum is appropriate and relevant to the local context (7 studies). Farmers observe 
benefits of FFS practices (5 studies). High levels of social capital among participants 
and tradition of collective action (3 studies). 

Barriers to and 
enablers of 
effectiveness and 
sustainability 

14 

studies 

Barriers: Diverging institutional incentives and objectives (3 studies). Conflicting 
agricultural policies (2 studies). Institutional legacy from top-down extension approaches 
(4 studies). Power of pesticide industry and continued links with the extension service (2 
studies). Lack of technical assistance and backstopping from researchers and 
extensionists (4 studies). 

Enablers: Active follow-up and continued support from implementing agency (11 
studies). FFS groups with consistent membership, good leadership, collective goals and 
a supportive group environment (4 studies). 

Barriers to and 
enablers of diffusion 
of knowledge and 
practices 

11 

studies 

Barriers: Complexity and experiential nature of FFS learning (5 studies). Farmers 
unable to observe FFS practices (2 studies). Farmers are not convinced of the relative 
advantage of FFS practices (2 studies). Socioeconomic differences between FFS 
participants and non-participants (1 study). Low levels of social capital and cohesion 
limiting communication (2 studies). 

Enablers: Concrete and relatively easy practices (2 studies). Farmers observe FFS 
practices (5 studies). Farmers perceive FFS practices to have relative advantage over 
existing practices (2 studies). High levels of social capital and social networks extending 
beyond FFS group (3 studies). Active promotion of FFS practices post-graduation (1 
study). 


Source: authors. 
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1 Background 


1.1 DESCRIPTION OF THE PROBLEM 


Agriculture is the main source of income for around 2.5 billion people in the developing 
world (FAO, 2003, p. 1) . Around 70 per cent of the extreme poor - or over 1 billion people - 
live in rural areas in low- and middle- income countries (IFAD, 2010, p. 233), most of whom 
rely directly or indirectly on agriculture for their livelihoods. Investment in agriculture has 
been shown to have beneficial impacts on agricultural growth (Fan &Rao, 2003) and, since 
the poorest population groups benefit significantly more from agricultural growth than from 
growth in other sectors of the economy, poverty reduction (United Nations, 2008; World 
Bank, 2007). 

The modernisation of farming practices in the 1960s and 70s during the "Green Revolution" 
improved agricultural yields substantially in those areas it reached and raised national 
production and food security (IFAD, 2001). However, a number of challenges emerged. The 
first problem was that poor farmers were being left behind, particularly in Sub- Saharan 
Africa where many were not reached by modernisation approaches. Those technologies that 
were promoted were not appropriate to the challenges facing smallholders in the African 
context, particularly women farmers (Inter- Academy Council, 2004). Second, where 
modernisation was successful, it was also associated with adverse environmental and health 
consequences, relating to water pollution, declining soil quality, soil erosion, pest resistance 
and loss of biodiversity (Van den Berg &J iggins, 2007). 

A particular problem emerged around the environmental and health consequences of 
chemical pesticide use. Chemical pesticides were so heavily promoted and publicly 
subsidised under the modernisation agenda that their overuse led to pesticide resistance and 
major outbreaks of insect pests in rice crops in Asia in the 1970s and 80s. In addition, 
prolonged exposure to pesticides was associated with chronic and acute health problems 
among rural residents (Pingali & Roger, 1995). Use of broad- spectrum insecticides in 
agriculture has also been linked to mosquito resistance to insecticides used in malaria 
control programs (Diabate et al., 2002; cited in Van den Berg &J iggins, 2007). 

It was increasingly recognised that different approaches were required to reach 
smallholders. These approaches needed to fulfil a broad range of objectives, including 
tackling the use of harmful pesticides (and other inputs), and reaching disadvantaged 
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farmers, in particular women, for whom appropriate technologies and methods of 
dissemination were needed. 


1.2 DESCRIPTION OF THE INTERVENTION 


From agricultural extension to adult education 

Agricultural extension and advisory services (hereafter extension services) comprise "the 
entire set of organisations that support and facilitate people engaged in agricultural 
production to solve problems and to obtain information, skills and technologies to improve 
their livelihoods" (Anderson, 2007, p. 6). Extension was traditionally viewed as a means of 
transferring technologies developed in research stations as well as farm management 
practices to farmers, and used top-down institutions of delivery, as characterised, for 
example, by the World Bank's Training and Visit System (Gautam & Anderson, 2000) . 

These traditional extension approaches were criticised for providing a "one size fits all" 
approach (Bimer et al., 2006), which failed to factor in the diverse socioeconomic and 
institutional environments faced by farmers, or involve farmers in the development of 
technology and practices appropriate to their contexts. Ultimately, extension is considered to 
have failed in achieving its main obj ective of farm productivity improvements and in 
reaching the poor, particularly in Africa (Anderson, 2007; Birkhaeuser, Evenson, &Feder, 
1991). 

Since the 1980s, the approach to reaching rural smallholder farmers has drawn increasingly 
on more participatory methods, which enable farmer self- learning and sharing, and also 
allow those facilitating farmer training, as well as agricultural researchers further upstream, 
to learn from the farmers (Bimer et al., 2006) . 1 More intensive training is considered 
necessary to disseminate complex messages, such as on integrated pest management. It may 
also empower farmers more generally to become problem-solving decision-makers, more 
adaptive and resilient to change. 

The IPM farmer field school 

Farmer field schools (FFS) have become a prominent participatory and learner- centred 
approach for agricultural development (see Appendix A for more information on the 
implementation of FFS around the world) . FFS originated in Asia as a means of improving 
farmers' analytic and decision-making skills, of which a key objective was to promote use of 
integrated pest management as an alternative to intensive pesticide spraying, which was 
severely damaging farm production, the environment and farmers' health. Integrated pest 
management (IPM) was developed in the 1960s and 70s (Kogan, 1998; cited in Kelly, 2005) 
and aimed to minimise pesticide use through use of more natural pest management 

1 There has been a similar evolution in the use of more bottom- up approaches to technology development 
through agricultural research, such as the local agricultural research committees (CIALs) approach (Braun et al., 
2000 ). 
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techniques. Integrated pest management methods promoted in FFS typically range from 
more simple practices, such as not applying pesticides in the first 30 days after planting ( "no 
early spray") and placing branches in rice fields for birds to perch on, to more complex 
methods that require in-depth agio- ecological and crop management knowledge, such as 
being able to differentiate beneficial from harmful insects, and creating a conducive 
environment for pest predators (Ricker- Gilbert et al., 2008). 

Figure 1 Components of a farmer field school intervention 


1 nception 


Training 


Dissemination 

• Recruitment and 


• Season-long training 


• Field days, exchange 

season-long training of 


attended by farmers 


visits 

FFS-facilitators 


• Facilitation through 


• Platform building (e.g. 

• Farmer group 


discovery- based group 


support to local 

formation 


learning (agro- 


networks) 

• Curriculum developed 


ecosystems analysis, 


• Training of farmer 

partially by farmers 


experimentation, group 


trainers 

• Other inputs: financial, 


dynamics, special topic) 



monitoring systems 






The core of FFS is experiential learning resulting from participation in the FFS process 
(Pontius et al., 2002) . The FFS intervention contains a handful of components which can be 
broadly categorised into three groups: the inception phase (field school development); the 
training phase (technology and curriculum); and the dissemination phase (components to 
promote diffusion of messages to non- participants (field days) or other farmer field schools 
(exchange visits), and institutionalisation of the schools through platform building and 
training of farmer trainers) (Figure 1) . A bottom- up participatory approach should underlie 
each component of the farmer field school intervention. Thus, the approach to group 
learning should be "discovery based" (Khisa, 2004) . The choice of curriculum should be 
based on priorities identified by farmers. And the aim of schools should not be just to 
disseminate I PM technology but also to focus on building problem-solving capabilities to 
"empower farmers to solve problems" for themselves (Kenmore, 1996). 

The standard FFS training involves a field- based season-long programme overseen by an 
FFS- facilitator, with weekly meetings near the plots of participating farmers (Pontius et al., 
2002) . Each FFS typically has 20 to 25 participants, with farmers working together in 
smaller groups. FFS- facilitators can be extension agents or selected graduates from previous 
FFS who undergo a training-of- trainers course tailored to equip them to facilitate field 
schools (Braun &Duveskog, 2008). The facilitators should use experiential, participatory 
and learner- centred educational methods, including experimentation through use of 
demonstration plots using the new practices based on the FFS technology (e.g. integrated 
pest management) with existing (business-as- usual) "farmer practice" plots, to enable 
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fanners to observe benefits (Pontius et al., 2002) . Standard field school curriculum includes 
agro- ecosystem analysis, involving presenting pictorially the factors which affect crops, 
special topics comprising locally specific problems, and activities to improve group 
dynamics. In addition, exchange visits to other field schools are organised to promote 
lessons learning, as are field days, in which participants present course material and the 
results of their studies to the broader community. Diffusion to the wider community may 
also involve encouraging participating farmers to engage in informal farmer- to- farmer 
communication, training of farmer trainers or approaches to institutionalise field schools 
locally. 

Broadening the farmer field school curriculum 

While all FFS are supposed to be based on the same process, the approach can be adopted to 
suit particular needs, crops or contexts (Pontius et al., 2002). Thus, as FFS have been 
promoted around the world, the technology has been modified to address the needs of 
farmers in different contexts, and applied to other food staples, vegetables and cotton 
(Appendix A) . In Africa, integrated production and pest management (IPPM) has been 
promoted. IPPM reflects a more "holistic" approach to improving production, in which pests 
and pesticide use are not necessarily the main production problems (Stathers et al., 2005). 2 
Other variants include integrated disease management (IDM), integrated crop management 
(ICM), integrated plant nutrient management (IPNM) 3 and integrated water and soil 
management (IWSM). The main types of technology incorporated in FFS projects are shown 
in Figure 2, which indicates that IPM and pesticide management are the most common 
technologies that have been promoted. 

As the approach has gained prominence, FFS have been implemented using different 
intervention components. FFS have also been implemented alongside complementary 
interventions to improve access to inputs, markets or collectivisation. Figure 3 shows the 
curricula and complementary interventions incorporated in farmer field schools design. 
Most project designs include FFS inception, but relatively few project documents indicated 
that the FFS training or dissemination involved key curriculum activities such as agro- 
ecosystems analysis. A minority of projects incorporated input supply and/ or produce 
marketing interventions. 


2 Drawing on the lessons of IPM, integrated vector management (IVM) is being applied in the health sector to 
combat malaria and other vector- borne diseases (van den Berg et al., 2007). This variant is beyond the scope of 
this review. 

3 See http:/ / www.fao.org/ agriculture/ crops/ core- themes/ theme/ spi/ scpi-home/ managing- 
ecosystems/ integrated- plant- nutrient- management/ en. 
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Figure 2 Technologies incorporated in farmer field school curricula (percentage of projects) 


■ Integrated Pest Management 
(IPM) 

■ Other pesticide management 

■ Soil management 

■ Integrated Production and Pest 
Management (IPPM) 

Integrated Crop Management 
(ICM) 

Other 

Integrated Crop and Pest 
Management (ICPM) 


Definitions: 

• Integrated pest management (IPM): IPM draws on an imderstanding of the life cycles of 
pests and their interaction with the environment to manage populations of pests 
economically while minimising risks to the environment or human health. 

• Integrated production and pest management (IPPM) : IPPM is a variant of IPM which 
has eveolved in Africa and which emphasises both the management of natural pests and 
the production of a healthy crop. 

• Integrated crop management (ICM) : ICM draws on an understanding of interactions 
between soil, the natural environment and biological pests or weeds to promote 
sustainable crop production. Example components include site selection, crop- specific 
production strategies, nutrient management and cover cropping. 

• Integrated crop and pest management (ICPM): ICPM combines chemical, biological and 
cultural pest control methods with crop management strategies. 

• Other pesticide management: This category includes non- specific references to chemical 
or pesticide management techniques. 

• Soil management: This category includes non- specific references to soil or crop 
management techniques. 

• Other: This category includes other variants (examples include integrated pest and vector 
management [IPVM] or integrated pest biosystem management [IPBM]), or other general 
references to management techniques. 


Source: Review of FFS project reports, conducted by the authors and documented in Appendix A. 
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Figure 3 FFS curricula and complementary components 



Inception: 

• Training of trainers: This entails providing theoretical and practical training to all FFS trainers. 
Training is usually season-long and typically includes facilitation and teaching skills, farming 
techniques and management skills. 

• Curriculum development: This involves planning a farmer field school's workplan, setting out the 
different modules to be studied, usually with some inbuilt flexibility. 

• Group formation: This involves setting up an organised group with specified goals such as a shared 
workplan, and which has an organisational structure such as a representative committee. 

Field school training and dissemination: 

• Agro-ecosystem analysis ( AESA) : This involves undertaking regular observations of crops to 
understand the interrelationship between soil, the natural elements and biological pests or weeds. 

• Special topic: This curriculum component involves the selection of a topic which is usually either 
suggested by the farmers themselves or is chosen as a response to field observations (e.g. the 
discovery of a pest in a field) . 

• Group dynamics: This activity is designed to develop leadership, problem- solving, discussion and 
communication skills. The emphasis is also on collaboration and the promotion of collective action. 

• Control plot: This involves the setting aside of control plots, which either receive no treatment at all 
or continue to be farmed as before. These plots serve as a baseline comparison for those where new 
techniques or technologies are implemented. 

• Other component: This category includes components specifically built for a particular group, such 
as hill farmers or ethnic minorities. Other additional components include community education and 
animal husbandry. 

• Field day: Field days are designed to promote the work going on in an FFS by inviting local farmers 
and people of influence to see what has been learned and the benefits that can be produced. 

• Farmer exchanges: This involves visits by FFS members to other schools with a view to observing 
their activities and exchanging ideas. 

• Platform building: This includes efforts to ensure that the benefits of FFS are sustained in the long 
term. Examples of such activities include organising farmers' dubs or building local networks which 
can ensure continued collective action. 

Complementary components: 

• Input supply: This entails the provision of farmers with seeds, tools or other equipment. 

• Marketing: This component involves providing farmers with training and help in setting up 
networks, transport and information- sharing geared towards marketing their products. 


Source: Review of FFS project reports conducted by the authors and documented in Appendix A. 
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FFS are seen as a way to overcome the traditional problems of extension in reaching and 
empowering disadvantaged farmers, particularly women in Africa (see, for example, Saito & 
Weidemann, 1990; World Bank &IBRD, 2009). In addition, the curriculum has also been 
broadened to tackle populations in particular contexts, such as J unior Farmer Field and life 
Schools (J FFLS) which have been implemented with youth across Africa and include HIV- 
risk reduction in addition to the agriculture components more standard to FFS (Braun & 
Duveskog, 2008). Other variants, such as business or marketing FFS, are intended to 
develop additional skills which can improve farmer livelihoods, or to adapt the original 
approach for an alternative type of farming. 


1.3 HOW FARMER FIELD SCHOOLS ARE SUPPOSED TO WORK 


Farmer field school programmes aim to build farmers' capacity and promote adoption of 
better practices, and consequently improve farmers' lives in terms of agricultural outcomes, 
health, environment and empowerment. The FFS approach has roots in Paulo Freire's ( 1970) 
approach to "dialogical education" using discovery- based learning. The FFS process aims to 
enable farmers to internalise the advantages of the improved agricultural practices through 
learning by doing and observation. FFS aim to empower farmers by encouraging them to 
develop skills in problem-solving using "scientific" methods of analysis, virile the group 
activities aim to empower farmers both within and outside their own communities and 
promote social cohesion through increased cooperation. 

A hypothesised causal chain for farmer field schools is depicted in Figure 4, which links 
farmer field school delivery inputs with final outcomes for FFS participants and for 
neighbouring non- participants who benefit through knowledge spillovers, via the 
intermediate outcomes of capacity building and technology adoption. The causal chain is 
rooted in transfer- of- technology models of extension (Bennett, 1975; cited in Funnell & 
Rogers, 2011) due to the objectives of many field schools to disseminate technology such as 
I PM. However, a key objective of FFS may also be to empower farmers cognitively through 
skills development, organisationally through group activities, and politically through 
collective action (Friis- Hansen, 2008). As recognised by Mandni and J iggins (2008, p. 540) 
FFS "have social goals beyond mere changes in pest- management techniques: goals that seek 
to position farmers as field experts, who collaborate with the extension staff to find solutions 
relevant to the local realities. FFS programmes emphasise farmers' ownership of 
development processes, partnership with other development agents, and group 
collaboration." 
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Figure 4 Farmer field school hypothesised causal chain 
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Underlying each link in the causal chain are the assumptions theorised to be important for 
changes to take place at each stage, and which therefore determine the extent to which 
impacts are likely to materialise in practice. These can be broadly grouped into intervention 
design and implementation characteristics, and local characteristics including those of 
fanners themselves. 

For instance, with respect to implementation, facilitators are a key input. Facilitators may be 
"traditional" extension agents who have received training in the FFS approach, meaning they 
are required to move away from the top-down approaches to which they are familiar, and 
adopt a more participatory, learner- centred approach (Feder et al., 2004a) . Facilitators 
should be adequately trained, involving season-long theoretical and practical training. 
Similarly, the relevance of the FFS curriculum to farmers - the extent to which the new 
practices are appropriate given inputs availability, and are observed to work compared with 
existing farmer practices (control plots which use standard approaches to pest management) 
- will likely influence farmers' attitudes and behaviour change. Indeed, Pontius et al. (2002) 
state that existing "farmer practice" plots should be a part of every FFS for comparison 
purposes. 
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Fanners also need to be trained adequately, so that they have attended sufficient meetings 
over the planting season, which means identifying the "right" farmers who are willing and 
able to participate in FFS training throughout the full season, and be able to implement FFS 
practices in their fields. Characteristics of local communities, such as heterogeneity in terms 
of land- and asset- holdings, ethnicity, education, gender roles and the degree of social 
cohesion, will determine the ability of the schools to reach appropriate beneficiaries, 
including disadvantaged farmers such as women. 

In the case of IPM field schools, diffusion to neighbouring farmers who have not attended 
formal training may be necessary for sustained adoption due to externalities associated with 
pesticide overuse - that is, where the social costs of pesticide use for the community exceed 
the private costs to the individual farmers. 4 Approaches to diffusion may be through 
informal farmer- to- farmer communication, one-off activities such as field day visits, or 
formal attempts to institutionalise community- based field schools and conduct training- of- 
trainers programmes for FFS alumni (Figure 1) (Pontius et al., 2002). In the absence of such 
approaches, characteristics of local communities may be important determinants of the 
degree of diffusion of knowledge and practices from participants to non- participants. 

The assumption that there will be some diffusion of IPM practices between farmers may not 
be an unreasonable one in principle for simple practices. However, FFS graduates may be 
limited in their ability to transmit all but the simplest of messages effectively to other 
farmers through informal means. 5 Guidelines from the Food and Agriculture Organization of 
the United Nations (FAO) on "community IPM" indicate formal approaches involving FFS 
alumni are considered necessary: "without post- FFS educational opportunities, there will be 
no community movement" (Pontius et al., 2002). Whether the diffusion mechanism is 
informal or formalised will, therefore, have implications for benefidary targeting (Feder & 
Savastano, 2006) . Without formal mechanisms, participants would ideally be selected if they 
have characteristics which will enhance diffusion, such as those respected in their 
communities and those with strong social networks. This may conflict with other objectives 
of FFS, such as targeting women farmers. 

Contextual factors, notably weather conditions, soil fertility, plant disease and climate 
trends, are obvious factors determining production and yields. The policy environment is 
also important, including whether FFS are implemented in the context of complementary 
agricultural policies, including those relating to input supply and marketing. Market prices 
and market access, both to purchase inputs and sell produce, determine the value of 
production and therefore farmer income. The price of inputs such as pesticide relative to 
(opportunity costs of) labour is also likely to determine adoption of IPM and other 


4 Lack of adoption of IPM practices by neighbouring farmers is theorised to curtail the effectiveness of the 
intervention, as pests from fields of non- adopters may re- infest the fields of adopters, eventually leading to 
disadoption of IPM by FFS participants (Feder et al., 2004b) . 

5 Evidence suggests that, in the case of technically complex issues or costly technologies, farmers prefer first-hand 
knowledge or advice from specialised information sources such as experts (Feder & Slade, 1984; cited in Feder et 
al. 2004b). 


34 


The Campbell Collaboration | www.campbellcollaboration.org 



technologies, where adoption involves substantial increases in demands on farmers' time. 
Indeed, FFS projects commonly incorporate complementary components, such as seeds or 
tools, settingup of farmer organisations and networks, and providing marketing training 
(Appendix A). 

Finally, it is also possible that the technologies promoted by FFS do not act to change yields 
(the amount of crop produced per unit of land area), but still act to improve income and net 
revenues (value of production less input costs) by reducing pesticide costs, provided these 
are not offset by any net increases in other costs such as labour in applying the new 
technologies. Moreover, in contexts where pesticides and pest management are not 
necessarily the key constraints to production, improvements in productivity may not 
necessarily arise from reduced pesticide use but as a result of adoption of the other practices 
being promoted, such as soil management. 


1.4 WHY THIS REVIEW IS NEEDED 


Since the 1980s there has been a decline or stagnation in public expenditure on agriculture 
in most developing countries (Akroyd & Smith, 2007). Likewise, the proportion of official 
development assistance (ODA) going to agriculture is estimated to have declined from 
around 20 per cent in 1979 to a low of 3.7 per cent in 2006, and has remained around 5 per 
cent since (Cabral &Howell, 2012). As noted in the World Development Report on 
Agriculture, "extension services, after a period of neglect, are now back on the development 
agenda. . . [but] more evaluation, learning, and knowledge sharing are required to capitalize 
on this renewed momentum" (World Bank, 2007, p. 175). Poverty reduction strategies in 24 
African countries also listed extension as a top agricultural priority (InterAcademy Council, 
2004; cited in Davis, 2006). Nevertheless, age-old questions in agriculture remain, including 
how to raise yields and farmer incomes, how to ensure environmentally sustainable 
development, and how to empower the poorest farmers and particular groups such as 
women farmers in developing skills in adoption and resilience to shocks. There is increasing 
criticism as to whether extension services are capable of achieving these broad objectives, or 
whether a more intensive approach is required such as that provided by the farmer field 
school initiative. 

Originally developed for rice crops in Indonesia in the 1980s by the FAO, fanner field 
schools have been introduced to at least 90 countries worldwide (Figure 5), by a range of 
organisations, producing 10- 15 million field school graduates by 2008 (Appendix A) . They 
are largely funded by multilateral development agencies and implemented by developing 
country governments and non-governmental organisations. Over half of all FFS projects 
have been based in Africa; however, the majority of beneficiaries (around 60%) have been 
Asian, indicative of the fact that some Asian FFS programmes have been implemented on a 
national scale. Figure 5 presents the growth in implementation of the FFS approach since the 
early 1990s, illustrating a marked increase in the number of projects in Asia and especially in 
Africa since 2000. 
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Figure 5 Coverage of farmer field schools in low- and middle- income countries (LMICs) 


Figure 1 LMIC Coverage of Farmer 
Field Schools 

■ 1935-1939 

■ 1990-1994 

■ 1995-1999 

■ 2000-2004 

□ 2005-2009 

□ 2010 onwards 

□ Start date N/A 




Source: Review of FFS project reports, conducted by the authors and documented in Appendix A . 


Figure 6 Cumulative number of farmer field school projects implemented 



projects for which a start date could be established. Source: Review of FFS project reports, conducted 
by the authors and documented in Appendix A. 


Hundreds of studies have evaluated farmer field schools. These studies provide conflicting 
conclusions on effectiveness. One particularly influential impact evaluation of the National 
IPM-FFS Programme in Indonesia concluded that "[t]he analysis, employing a modified 
'difference- in- differences' model, indicates that the program did not have significant impacts 
on the performance of graduates and their neighbors" (Feder et al. 2004a, p. 45). The study 
appears to have been highly influential in the policy community, including contributing to 
the World Bank pulling out of the Global IPM Facility multi-donor trust fund (Kelly, 2005) . 

Reviews drawing on evaluations from more than a single context have tended to be rather 
more positive. Van den Berg (2004) synthesised 25 IPM-FFS evaluation studies, concluding 
that, "Studies reported substantial and consistent reductions in pesticide use attributable to 
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the effect of training. In a number of cases, there was also a convincing increase in yield due 
to training . . . Results demonstrated remarkable, widespread and lasting developmental 
impacts" (p. 3). Van den Berg and J iggins (2007) also argued that FFS have had additional 
benefits to those of I PM, including facilitating collective action, leadership, organisation and 
improved problem-solving skills. 

However, Tripp et al. (2005) noted the lack of rigorous evidence on the effectiveness of the 
approach, despite the sizeable investments in FFS in Asia. Reviewing seven studies, they 
concluded that while "the FFS approach has undoubtedly succeeded in lowering insecticide 
use in a number of Asian rice examples, judgments on its overall impact await further study" 
(p. 1,711). They also found little evidence to suggest effective diffusion of IPM knowledge 
between FFS participants and non- participants, nor sufficient evidence to conclude that FFS 
groups continue on their own. 

In addition to the debate about effectiveness, the scalability and financial sustainability of 
FFS has also been questioned. FFS are a particularly intensive intervention, with high costs 
in terms of both facilitation and opportunity costs of beneficiaries' time. Leading authors 
from the literature have therefore noted that FFS are unlikely to be a solution to problems of 
extension delivery, and only scalable under certain drcumstances (Braun et al., 2006; Davis, 
2006). Quizon et al. (2001) noted the lack of fiscal sustainability as a generic problem 
affecting large-scale public extension services, concluding that FFS face the same issues as 
other approaches. The cost per fanner is likely to be high compared with agricultural 
extension approaches and the evidence from Indonesia suggested a low rate of informal 
diffusion from direct beneficiaries of the schools to neighbours. 6 The authors suggested that 
as the situation for farmers, in terms of political power, governance systems and day-to-day 
interactions among farmers, is quite similar in many other developing countries in Asia and 
Africa, the results were relevant for discussions of similar extension activities in these areas. 
They also warned that, while pilot proj ects might indicate the viability of the FFS approach 
in certain circumstances, the issue of fiscal sustainability becomes particularly relevant when 
scaling up. 

However, Braun and Duveskog (2008) argued that the relative cost-effectiveness of FFS 
should be put in the context of rural adult education rather than extension "when FFS are 
regarded as a form of public investment in farmer education to tackle rural poverty - and 
hence as a tool for achieving the Millennium Development Goals" (p. 19) . Van den Berg and 
J iggins (2007) also noted that discussions on the fiscal sustainability of FFS should take into 
consideration who will pay for the externalities of pesticide use. 

The existing reviews provide some suggestive evidence of the effects of FFS, but come to 
widely different conclusions in a hotly debated, policy- and operations- oriented literature. 


6 In Bangladesh, Ricker-Gilbert et al. (2008) estimated per capita costs of fanner field schools (including 
opportunity costs of farmer and trainer time) at over ten times those of agricultural extension approaches 
including demonstration field days and extension agent visits. 
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However, while single studies are unable to provide a complete picture of the evidence, none 
of the existing reviews draws on a systematic search for all available quantitative and 
qualitative studies, or applies inclusion criteria or approaches to critical appraisal sufficiently 
transparently. In addition, the conclusions of these reviews are based on significance- based 
vote counting, rather than sample- weighted meta- analysis of effects . 7 This systematic review 
thus aims to provide a systematic and exhaustive search, together with a comprehensive and 
unbiased synthesis of the existing evidence on FFS. 


7 There are meta-analyses on the effects of agricultural extension more generally, notably Alston et al. (2000) . 
However, these studies frequently draw on economic appraisals based on weak counterfactuals. 
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2 Aims and Report Structure 


The primary obj ective of the review is to synthesise evidence on the effectiveness of 
interventions identified as "fanner field schools" and conducted in low- and middle- income 
countries. The review aims to provide an integrated synthesis based on analysis of two main 
research questions: 

• Review question ( 1) : What are the effects of farmer field schools on final outcomes 
such as yields, net revenues and farmer empowerment (review question la), and 
intermediate outcomes such as knowledge and adoption of improved practices (e.g. 
reduced use of pesticides) (review question lb) in low- and middle- income countries? 
What are the effects on non- participating neighbouring farmers (review question lc)? 

• Review question (2) : What are the enablers of and barriers to FFS effectiveness, 
diffusion and sustainability? 

We followed Campbell and Cochrane Collaboration approaches to systematic reviewing 
(Shadish &Myers, 2004; Hammerstrom et al., 2010; Becker et al., n.d.; Shemilt et al., 2008; 
Higgins & Green, 2011), and drew on theory-based impact evaluation (White, 2009). 8 To 
answer review question ( 1), we systematically collected and synthesised all relevant and 
available quantitative evidence from impact evaluations of FFS programmes. We used 
narrative review and, where possible, statistical meta- analysis, presenting outcomes along 
the theory of change (Figure 4), from intermediate outcomes such as capacity building, 
technological adoption and diffusion to indirect beneficiaries, to final outcomes such as 
agricultural yields, household income and other indicators of wellbeing in the areas of 
health, environment and self-esteem, examining heterogeneity in findings narratively and in 
meta- regression. Studies that measured outcomes at any point along the causal chain were 
eligible for inclusion, and evidence is presented outcome by outcome for all studies that 
reported each particular outcome. 

Farmer field schools are complex interventions implemented using different methods of 
delivery in a range of different contexts. For the review to be useful to a broader group of 
decision-makers, we extended it by including qualitative studies to address review question 
(2), focusing on underlying factors that determine or hinder the effectiveness of FFS, 


8 3ie's approach to systematic reviewing is provided in Waddington et al. (2012b) and Snilstveit (2012). 


39 


The Campbell Collaboration | www.campbellcollaboration.org 


following Noyes et al. (20 11). 9 Most quantitative impact evaluations do not include research 
addressing this question so, as outlined in the following chapter on Approach, we adopted 
different inclusion criteria for studies addressing this question. We conducted the two 
syntheses in parallel, before integrating the findings in a narrative synthesis using the theory 
of change (Figure 4) as a framework for the analysis. 

We also reviewed FFS project documentation to provide a global portfolio review of 
implementation experiences (Appendix A). 10 

Figure 7 provides a detailed outline of the review process for review questions ( 1) and (2) . 
The strategy for systematic searches and synthesis, elaborated in Chapter 3, was published in 
the study protocol (Waddington et al., 2012a). As is standard in reviews of development 
topics, a large number of initial search results were returned. Of the 28,500 potentially 
relevant papers identified, over 9,000 were identified from databases, 18,000 from Google 
or Google Scholar, 65 from bibliographic searches of reviews and 29 from contact with 
organisations and researchers. After applying inclusion criteria, 92 distinct impact 
evaluation studies (from 134 papers) and 20 qualitative studies (27 separate papers) were 
included in the review. A total of 337 FFS interventions were included in the portfolio review 
(Appendix A). 


9 Our methodology was also informed by Chapter 20 in the Cochrane Handbook (Noyes et al, 20 11), the 
additional guidance developed by the Cochrane Qualitative Methods Group (Hannes, 20 11; Noyes & Lewin, 20 11) 
and the increasing number of examples of systematic reviews in international development based on or 
incorporating qualitative evidence (e.g. Munro et al., 2007; Williamson et al., 2009; Berg&Denison, 2012). 

10 We also conducted an additional systematic review of data on targeting contained in the full text studies we 
obtained for the systematic review research questions detailed in this report. We synthesised a small part of the 
information on targeting in the moderator meta- analysis section of this report, although readers are encouraged 
to access the report of the systematic review of targeting for the full analysis (Phillips et al., 20 14) . 


40 


The Campbell Collaboration | www.campbellcollaboration.org 



Figure 7 Review process 
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The remainder of the report is structured as follows. Chapter 3 on Approach summarises the 
review methodology. Chapter 4 presents the results from the synthesis of impact evaluative 
evidence, including quantitative meta- analysis, indicated on the left-hand side of Figure 7. 
Chapter 5 presents the results of the synthesis of barriers and enablers based on qualitative 
evidence, indicated on the right-hand side of Figure 7. Chapter 6 then presents the results of 
the integration of the two syntheses. Chapter 7 provides implications for policy, practice and 
research. 
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3 Approach 


3.1 STUDY SELECTION CRITERIA 


Studies included in the review met the following selection criteria. 

Types of interventions 

Studies needed to report specific "farmer field school" interventions. Interventions were 
identified as farmer field schools if they contained both of the following components: 

• Involved intensive, facilitated group training, normally involving season-long weekly 
meetings and use of control plots farmed using standard farmer practices. 

• Provided information on holistic techniques to inputs use, such as reducing use of 
pesticides and insecticides, improved use of fertiliser, or other production practices 
and disease control methods such as integrated pest management, integrated 
production and pest management, integrated crop management and integrated 
disease management. 

It was often difficult to identify the exact components of the FFS, so the approach we took 
involved including interventions identified as "farmer field school" in the report, even where 
the FFS may have been designed or implemented differently. Studies were eligible that 
investigated provision of FFS alone or combined FFS with other intervention components, 
such as input or marketing support. 

Types of participants 

The review included farmers growing arable crops ( "temporary" crops including food and 
cash crops) and permanent crops (such as cocoa, coffee and tea), living in developing (low- 
or middle- income) countries, as defined by the World Bank, at the time the intervention was 
carried out. Studies were included which collected and reported on data at the farm or 
household level. The review excluded programmes for livestock farmers, who received 
different types of training than crop farmers, and those for farmers based in high- income 
countries where the challenges faced in terms of poverty, land size, crops, and agro- 
ecological and environmental contexts are usually very different. 
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The review examined effects on two groups of beneficiaries: the farmers who participated 
directly in the field school and non- participating neighbour fanners who lived in the same 
communities as field school graduates and may have been exposed to the approach through 
their interactions with FFS- trained farmers (spillover effects) or more formal dissemination 
methods; effects for FFS fanners and neighbour farmers are analysed separately. We report 
information from studies that assessed outcomes for either type of beneficiary. 

Types of comparisons 

If fanners in low- and middle- income countries do have access to sources of information 
about agricultural practices, it is usually through visits from government or private- sector 
agricultural extension agents, through observation of public agricultural demonstration 
plots, or through agricultural extension provided by the private sector. Public extension may 
take the form of centralised or more decentralised systems (Bimer et al., 2006) . We included 
studies which compared fanners receiving FFS education with comparison groups who 
received no intervention, or agricultural extension services from another source, including 
I PM (or equivalent) training. We collected relevant information on the intervention received 
by comparison groups, and where possible calculated FFS effects across appropriate groups. 
To take one example, in Godtland et al. (2004), FFS fanners were chosen from among 
fanners who had previously received another extension implemented by a non-governmental 
organisation (NGO) - the Andino programme. Godtland and colleagues compared FFS plus 
Andino with farmers receiving the Andino programme only and with fanners not receiving 
either extension programme. We therefore used the FFS- Andino versus Andino comparison 
to calculate the net impact of FFS. Where we did suspect differential access to another 
relevant agricultural intervention in the comparison group, we conducted sensitivity 
analysis. 

We would have preferred to limit the review to studies with comparison groups which were 
separated geographically from treatment groups to avoid possibilities of spillover 
(contamination) . However, many studies reviewed did not report sampling procedures in 
sufficient detail to assess the geographic separation of groups. Thus, we included separate 
and non- separate comparisons, and assessed the likelihood of spillover effects in risk of bias 
analysis. We also conducted sensitivity analysis for potential spillover effects 
( contamination) . 

Types of outcome measures / data collected 

Intermediate and final outcomes 

We included all eligible studies to answer review question ( 1), irrespective of whether they 
reported impacts on intermediate or final outcomes. The review focused primarily on 
agricultural outcomes, including agricultural yields (production per unit of land), and net 
revenues (profits per unit of land) as indicated in the study protocol ( Waddington et al., 

20 12a) . We examined secondary intermediate outcomes including farmer knowledge and 
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capacity and adoption of new approaches (e.g. reduction in pesticide use) . We also examined 
secondary final outcomes including health, environment and empowerment outcomes such 
as feelings of self-esteem. Table 1 shows the different ways of measuring outcomes reported 
in included studies. Data on outcomes along the theory of change are reported outcome by 
outcome, for all studies reporting that particular outcome. Unless otherwise specified, 
outcome data are not reported along the theory of change by individual study. 

Other data 

To answer review question (2) we included evidence on barriers to and enablers of FFS 
effectiveness, diffusion and sustainability. This included process and implementation 
information together with measures of beneficiaries' attitudes and experiences with FFS. 


Table 1 Outcomes reported in effectiveness studies 


Outcome type 

Detailed outcome variables collected for FFS participants and neighbours 

Intermediate outcomes: 

Knowledge 

Test score: mean value across farmers 

Probability: percentage of farmers with improved knowledge 

Number of technologies known: average number of specific knowledge reported 

Adoption: pesticides 

Pesticide use: volume/weight per unit of land area; average number of sprays per season; 

percentage of farmers using pesticides 

Pesticide costs: monetary value per unit of land area 

Adoption: other practices 

Percentage of farmers using beneficial techniques 

Index of adoption: mean score across farmers 

Number of beneficial techniques adopted: mean across farmers 

Time spent undertaking field observation 

Final outcomes: 

Agricultural outcomes 
(primary outcomes) 

Yield: weight of production per unit of land area; monetary value of yield (monetary value of 
production per unit of land area) 

Total income: monetary income, sales (quantity of production sold at market) 

Net revenue: profits per unit of land area (monetary value of yield less input costs per unit 
of land area) 

Health outcomes 

Incidence of respiratory difficulties, eye irritation, stomach ache, blurred vision 

Probability of pesticide poisoning 

Annual expenditure on health per household 

Environmental outcomes 

Environmental impact quotient (E IQ) score 
Soil fertility measurement 

Empowerment outcomes 

Indices of self-esteem: feeling capable of solving problems, comfortable giving opinion, 
participating in community, collective action, making demands on extension staff 
Perception of empowerment 
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Study design and methods of analysis 

Review question ( 1) : What are the effects of farmer field schools on intermediate and final 
outcomes, for FFS partidants and neighbour farmers? 

Studies eligible for indusion in the quantitative synthesis used experimental or quasi- 
experimental study designs. Study designs which collected longitudinal data at baseline and 
endline and those using cross-sectional (endline) data only were induded. In addition, data 
needed to be collected at the farm or household level contemporaneously in both groups. 
Studies that used the following methods of allocating FFS to partidpants were eligible: 

• allocation rules based on prospective randomised or quasi- randomised (e.g. 
alternate) assignment (randomised controlled trials or RCTs, and quasi- RCTs); 

• assignment based on other known allocation rules, inducting a threshold on a 
continuous variable (regression discontinuity designs or HDDs) or exogenous 
variation in the treatment allocation ( "natural experiments"); 

• assignment based on other rules, including self- selection by programme planners or 
partidpants, provided data were collected contemporaneously in a comparison group 
(non- equivalent comparison group design), or where at least three data points were 
collected for FFS partidpants both before and after a discrete intervention (six- 
period interrupted time series or ITS 11 ). 

We induded studies which used statistical matching (e.g. propensity score matching or PSM, 
or covariate matching), regression adjustment (e.g. difference- in- differences or DID, and 
single difference regression analysis, instrumental variables or IV, estimation and Heckman 
selection models), as well as other cross-sectional or longitudinal designs which used less 
rigorous approaches. Given the breadth of designs included, we conducted rigorous 
assessment of internal validity based on risk of bias categories (see below) . 

Excluded studies are those which did not use a comparison group design, or employed less 
than a six- period ITS design. For example. Tin (2009) used a pre-test post- test design with 
no comparison group, and Armen et al. (2009) collected post- test data among field school 
participants only (see Appendix B) . 

Review question (2) : What are the enablers of and barriers to FFS effectiveness, diffusion 
and sustainability? 

We included qualitative studies and studies using descriptive statistics which met the 
following criteria: 


11 We followed Effective Practice and Organisation of Care (EPOC, n.d.) in adopting these criteria, based on the 
logic that at least three data points either side of the cut- off are needed to identify a trend. 
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1. reported on interventions as identified as "farmer field schools" although not 
necessarily the same interventions as those included in the review of effects (review 
question 1); 

2. assessed determinants of service delivery quality, knowledge acquisition, adoption of 
technological improvements, diffusion, or sustainability (either directly or indirectly 
- for example, studies that were relevant to addressing barriers to and enablers of 
FFS effectiveness); 

3. were based on primary data collected from clients, FFS facilitators, extension agents 
or experts analysed using qualitative methods or descriptive statistics; 

4. reported some information on all of the following: the research question, procedures 
for collecting data, sampling and recruitment, and at least two sample characteristics. 

We adopted a two- stage approach to inclusion of the qualitative studies, which, in addition 
to removing studies based on the usual relevance criteria (intervention, population, 
relevance to research question, study type and location), removed studies of particularly low 
quality in the first round (Thomas et al., 2003; Spencer et al., 2003), usingthe criteria set 
out in point 4 above. We then assessed the quality of the included studies using a detailed 
quality appraisal checklist in the second round, as described below. 

Given the limited reporting of programme and contextual characteristics in the impact and 
qualitative evaluation literature, in the final stages of the review we systematically searched 
for implementation documentation (see Appendix A for details), collecting data on project, 
programme and implementation characteristics which we linked to the impact evaluations in 
order to conduct more in-depth analysis of moderators. This analysis was conducted a 
posteriori. 


3.2 SEARCH STRATEGY 


Searching the social science literature can be challenging as it is not as well indexed as the 
medical literature. We developed the search strategy based on the guidance provided in 
Hammerstnam et al. (2010) and using "pearl harvesting" methods (Sandieson, 2006). 

We searched a range of different databases, including general social science databases, 
agriculture subject- specific databases and libraries, as follows: AgEcon, CAB Abstracts, Web 
of Knowledge (Social Sciences Citation Index and Social Science Conference Proceedings), 
International Bibliography of the Social Sciences, EconLit and the US National Agricultural 
Library (Agricola) . For the updated searches, we additionally searched an EBSCO multifile 
group of databases: Academic Search Research and Development, Africa- Wide Information, 
Business Source Complete, Soclndex. The fist of databases, together with the time of original 
search and update is provided in Appendix B. Detailed search strategies, together with the 
number of hits for each database, can be provided by the authors on request. 
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We used the following basic search strategy, adapted for each database to include thesaurus 
terms where these were available: 


"farmer* field* school*" OR (("integrated" AND "management") AND ("field* school*" or 
"farmer* field*")) 

The following example of a full search strategy is for CAB Abstracts using the Ovid platform 
incorporating both text words and thesaurus terms: 

1. (integrated control or integrated pest management or crop management). sh. 

2. (((integrated adj (production or management or pest or nutrient)) or crop 
management) .ti, ab. 

3. lor 2 

4. on-farm training.sh. 

5. (field school* or farm* school* or farmer* field* or (farmer* adj field* adj school*) ).ti,ab. 

6. (practical education or extension education or education programmes or community 
education or agricultural education or inservice training or vocational training or innovation 
adoption). sh. 

7. extension/ 

8. (participatory extension or agricultural advisory or agricultural extension or rural 
extension) ,ti,ab. 

9. 6 or 7 or 8 

10. exp developing countries/ or exp africa/ or exp asia/ or exp south america/ or exp central 
america/ or exp latin america/ or exp pacific islands/ or exp middle east/ or mexico/ 

11 3 and 9 and 10 

12. 5 and 9 and 10 

13. 4 or 11 or 12 

To ensure maximal coverage of unpublished literature, we searched J OLIS, BLDS, IDEAS, 
the Networked Digital Library of Theses and Dissertations, Index to Theses, the ProQuest 
dissertation database and the 3ie impact evaluation database, adapting the search strategy 
above for each database. We also searched Google Scholar, screening the first 1,000 hits. In 
addition, we searched the websites of a large number of international organisations, 
development agencies and non-governmental organisations active in the sector. Details 
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about the procedures followed when searching each website, together with the number of 
hits for each database, can be provided by the authors on request. 

We screened the bibliographies of included studies and existing reviews for eligible studies. 
We also handsearched development journals and identified and contacted key researchers 
and organisations working in the field of agricultural extension, as specified in the study 
protocol. All searches were updated in October 20 12. Further details are provided in 
Appendix B, which presents the full search strategy and dates of searches. In the final stages 
of the review we also conducted a systematic search for proj ect implementation 
documentation, documented in Appendix A. Titles and abstracts were screened against the 
inclusion criteria and relevant records were downloaded into the reference management 
software EndNote. The initial records search was conducted by two reviewers, who were 
over- inclusive to ensure relevant studies were not omitted because sufficient information 
was not reported in title or abstract. Two reviewers independently reviewed the downloaded 
abstracts in more detail to determine which papers should be retrieved and reviewed at full 
text, and included in the review. 


3. 3 DATA COLLECTION AND ANALYSIS 


Selection of studies and data extraction 

Two independent reviewers assessed the full text papers against the inclusion criteria. 
Discrepancies were resolved by consensus or by a third author if needed. Owing to the broad 
study design eligibility for impact evaluations, there was only one disagreement on inclusion, 
relating to a participatory agricultural extension project reported in Smale et al. (2010), 
which was judged as ineligible due to lack of relevance. Five reviewers extracted data from 
included studies. 

J H extracted data from included papers and then two reviewers (J H and HJ W) made risk of 
bias assessments and effect size calculations referring to the original papers. For studies 
assessed as having medium risk of bias, effect sizes were calculated by bother reviewers 
using Microsoft Excel. In all other instances, J H extracted and calculated effect sizes and 
HJ W reviewed a random selection of ten papers. There were no disagreements on risk of 
bias status or effect size calculation for medium- risk- of- bias studies. A Campbell 
Collaboration peer reviewer disagreed with the positive assessment of 'other risk of bias' for 
many studies due to lack of blinding of outcome assessors and data analysts, which we 
subsequently amended by downgrading this risk of bias criterion for relevant studies. There 
were five effect size disagreements in total (from three papers) for the high- risk-of- bias 
studies. Disagreements were resolved through an audit of data extraction spreadsheets by 
the lead reviewer ( HJ W) . 

Two reviewers (BS and MV) independently conducted the critical appraisal of studies 
included in the qualitative synthesis using an adapted version of the CASP checklist as 
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described below, with any disagreements resolved by a third author (PD). We used the FFS 
programme theory, as well as Rogers' theory of diffusion (2003) to develop a data extraction 
sheet for the studies included in the qualitative synthesis a priori. The data extraction sheet 
is provided in Appendix B. Three reviewers (BS, DP and MV) extracted information from 
included studies, using NVivo and Excel. Two reviewers then checked the coding of all the 
included studies, extracting any additional information missed by the first coder. 

Critical appraisal 

Review question ( 1) : Critical appraisal of studies of effects 

Studies were critically appraised according to risk of bias in internal validity and external 
validity (generalisability), and publication bias. The assessment of risk of bias was based on: 
1) quality of attribution methods (addressing confounding and sample selection bias); 2) the 
possibility of spillovers to farmers in comparison groups; 12 3) outcome and analysis 
reporting biases; and 4) other sources of bias (Appendix C). Risk of bias was assessed on 
both study design and implementation of the evaluation methodology. "Low- risk- of- bias" 
studies were identified as those in which clear measurement of and control for confounding 
was made, including selection bias; there were no sources of unobserved confounding which 
were likely to affect our degree of confidence in the findings; intervention and comparison 
groups were described adequately (in respect of the nature of the interventions being 
received) so that risks of spillovers or contamination were small; and where reporting biases 
and other sources of bias were unlikely. We rated the likelihood of spillover effects as low 
when comparison fanners were living in a different village from the farmer field school 
participants. 

Studies were identified as "medium risk of bias" when there were moderate threats to 
validity of the attribution methodology, or likely risks of spillovers or contamination (arising 
from inadequate description of intervention or comparison groups or possibilities for 
interaction between groups such as when they are from the same community), or possible 
reporting biases. 

"High risk of bias studies" were all other studies, including those where the study design was 
of questionable internal validity (such as those where comparison groups were not matched 
on observables, differences in covariates were not accounted for in multivariate analysis or 
where there were serious threats to the validity of the statistical procedure used to deal with 
attribution), or where there was evidence for spillovers or contamination, and reporting 
biases were evident. Two reviewers (J H and HJ W) undertook the critical appraisal, the 
results of which are presented in Appendix F. 13 


12 Note that, in contrast, spillovers to non- participant neighbour farmers are desirable for the intervention, and 
are assessed by the measured effects reported on these groups, in separate meta-analysis. 

13 Full details of the critical appraisal assessment are available on request from the authors. 
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The assessment of external validity of studies (generalisability) was based on two sources of 
information. Firstly, data were collected on the sampling methodology to assess whether the 
whole population or a random or purposive sample of FFS farmers and neighbours were 
covered. Second, the socioeconomic and demographic characteristics of farmers in the 
included studies were compared to average characteristics of farmers to assess whether FFS 
farmers were more or less representative of farmers in developing countries. We attempted 
to reduce publication bias by searching for and including unpublished studies in the review. 
However, we also assessed the likelihood of file- drawer effects resulting from selective 
outcome reporting and conducted statistical tests for under-reporting of small sample 
studies with negative or insignificant findings using funnel plots and Egger's (Egger et al., 
1997) tests for those outcomes with more than ten observations. 

Review question (2): Critical appraisal of studies examining barriers and enablers 

We assessed the quality of included studies using an adapted version of the Critical Appraisal 
Skills Programme checklist (CASP, 2006), making judgments on the adequacy of reporting, 
data collection, presentation, analysis and conclusions drawn. The checklist is presented in 
Appendix C. In accordance with our inclusion criteria we filtered out studies of particularly 
low quality (Hannes, 20 11) and studies where questions 1- 5 on the checklist were assessed 
as "No" were excluded at this stage. The critical appraisal was conducted by two reviewers 
independently (results reported in Appendix F) . Any discrepancies were resolved through 
discussion, with a third person acting as an arbitrator. 

Measures of treatment effect 

Calculation of effect size 

We calculated effect size estimates, standard errors and 95 per cent confidence intervals 
using data provided by included studies, where possible. We report two types of effect sizes 
in this review. All outcomes were measured using continuous variables. We calculated 
response ratios to measure effects, which are expressed as the difference in outcome in the 
intervention group as a proportion of the outcome in the comparison group. The response 
ratio (RR) is centred around 1, which is the point of "no effect"; the distance above and below 
the no- effect point are translatable as percentage changes in the outcome in the treatment 
group over the comparison, giving the same interpretation as a risk ratio. Thus, an RR of 1. 10 
translates as a 10 per cent average increase in the treatment condition, while an RR of 0.90 
would translate as a 10 per cent average reduction. 

The response ratio has the twin advantages of ease of interpretation and ease of calculation, 
since it requires less information than the standardised mean difference to compute. 
However, the use of the response ratio is subject to limitations for some variables, since it is 
only meaningful when the outcome is measured on a true ratio scale that has a natural zero 
point, although is unlikely to be equal to zero in practice (Borenstein et al., 2009). Thus, we 
calculated RR for some of the outcomes, including pesticide use, environmental measures 
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estimated from pesticide use, production yield and revenues, and variables measuring 
probabilities such as disease incidence, and empowerment. In the cases of knowledge test 
scores and indices of adoption of practices, we estimate the Hedges' g (sample size 
corrected) standardised mean difference ( SMD) which measures the effect size in units of 
standard deviation of the outcome variable. 

We have used appropriate formulae to calculate zero- order and partial effect sizes, 
respectively, from (unadjusted) bivariate and (adjusted) multivariate specifications. 
Appendix D provides details of all effect size calculations. 

However, caution is needed regarding the synthesis since most of the studies included in the 
review of effects, including all of the studies assessed as being of medium risk of bias, have 
used multivariate specifications in their analyses. As such, the meta- analysis synthesises 
both partial and bivariate effect sizes. However, effect sizes are only strictly comparable in 
studies employing a common model, meaning that "suitable proxies for the same constructs 
[i.e. the outcome variable, treatment variable and covariates] are included in all the studies 
being synthesized" (Keef & Roberts, 2004, p. 103). This is due to the degree of correlation 
between the variables included in the model, over and above any problems due to model 
misspedfication. In other words, the partial effect size based on the regression coefficient 
measures the treatment effect "holding all other variables constant". It is therefore 
measuring a different quantity to the bivariate relationship where the treatment effect is 
correlated with the other explanatory variables. The partial and bivariate effect sizes are 
equal only where coefficients of explanatory variables are the same in treatment and 
comparison groups (the "constant slopes" assumption), as would be indicated, for example, 
by insignificant interaction terms between treatment variable and covariates (ibid.) . 

As indicated in Appendix D, several solutions have been proposed to address this problem. 
Becker and Wu (2007) notably propose drawing on the variance- covariance matrix from the 
included studies to estimate generalised least squares meta- regression analysis. However, 
there are several problems in applying Becker and Wu's solution. First, not all multivariate 
models control for the same covariates, nor should models estimated for different study 
designs using data collected in different contexts necessarily do so. And where they do use 
common covariates, the variance- covariance matrices are seldom reported and difficult to 
obtain. Indeed, we were not able to identify any included studies in this systematic review 
which reported the full variance- covariance matrix. 

We have opted not to ehminate studies which did not report bivariate effect sizes from the 
synthesis, due to the loss of information this would entail and the likely high risk of bias in 
included studies which did not use multivariate specifications (since none of the studies used 
randomised assignment or similar design- based methods to reduce biases) . While the risk of 
bias assessment evaluated likely specification errors, included studies did not report 
diagnostic test statistics for multicolhnearity or results of models estimated with full 
interaction terms. Our RR and g effect size calculations therefore implicitly assume zero 
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correlation between treatment effect variable and other covariates in the model. However, 
many of the commonly used covariates (socioeconomic status, demographic characteristics 
and location) in the farmer field schools literature are likely to be correlated with the 
treatment, violating the constant slopes assumption. With the aim of partially ameliorating 
this problem, where different studies did report bivariate and partial effect sizes for a 
particular outcome, we have conducted sensitivity analysis to examine evidence for 
systematic differences (with the corollary that these are indirect comparisons across studies, 
not direct comparisons of bivariate and partial effect sizes within the same study) . 

Dependent effect sizes 

We only included one effect estimate per study in a single meta- analysis. Where multiple 
outcomes were reported from alternate specifications, we selected the specification 
according to likely lowest risk of bias in attributing impact, for example the most 
appropriately specified regression equation, which may in some instances be the least 
parsimonious. In some cases, where studies reported multiple dependent effect sizes, for 
example according to different outcome sub-groups (such as the paper by Ricker- Gilbert et 
al., 2008, which reported simple, intermediate and complex knowledge), we calculated a 
"synthetic effect size" based on the sample- weighted average, using appropriate formulae to 
recalculate variances according to Borenstein et al. (2009, chapter 24), making covariance 
assumptions as necessary (more information is available in Appendix D) . We used the same 
approach where studies reported multiple effect sizes according to different follow-up 
periods, although we also discuss in the report differences in reported follow-ups for those 
studies which did so. Where studies reported multiple effect sizes according to sub-groups of 
participants, we report data on relevant sub-groups separately (as in the case of FFS 
participants and neighbours) . 

We report data in the meta- analysis according to the paper in which the effect size 
originated. However, we attempted to avoid synthesising dependent effect sizes from 
multiple studies in any single meta- analysis by linking papers prior to analysis, and 
conducting sensitivity analysis as necessary. This was deemed necessary in the case of Feder 
et al. (2004) and Yamazaki and Resosudarmo (2008), which replicate analysis of the same 
data. In a number of cases, information has been collected on the same programme at the 
same or different periods of time, but in most cases this did not cause problems for the 
analysis since the papers either reported data on different outcome constructs which are 
eligible for inclusion in separate meta- analyses, or they did not report sufficient information 
on standard deviations or outcome means to extract effect sizes. 

For example, David and Asamoah (2011) reported knowledge outcomes based on a survey of 
farmers in 2007-08 in Ghana, while Gockowski et al. (2010) reported on adoption and yields 
for farmers in the same programme in 2005; David (2007) and Wandji et al. (2007) also 
reported outcomes in Cameroon for the same multi- country programme in West Africa as 
Gockowski et al. (2005), although the latter only reported impacts on adoption and yields in 
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the project areas of Nigeria. Chietal. (1999) and Murphy et al. (2002) both collected data on 
the FAO National I PM Programme in Vietnam, but only Murphy reported sufficient 
information to estimate the effect size. Mutandwa and Mpangwa (2004) and Maumbe and 
Swinton (2003) both reported on cotton IPM-FFS in Sanyati District, Zimbabwe, but only 
Mutandwa and Mpangwa presented statistical information. Dankyi et al. (2005) and 
Carlberg et al. (2012) reported different outcomes for the same programme, as did Reddy 
and Suryamani (2005) and Pananurak (2010) for the same Indian programme; DANIDA 
(2011) and Islam et al. (2006) reported data on the same Bangladesh programme but for 
different outcomes and different years. Similarly, Friis- Hansen and Duveskog (2012) and 
Friis- Hansen et al. (2004) collected data on empowerment and adoption outcomes, 
respectively, for the FAO's East African IPPM-FFS Pilot Project in Uganda, while Davis et al. 

( 20 12) collected data on agricultural outcomes for the expansion phase of the same proj ect. 
Torrez et al. ( 1999) reported on the pilot of the IPM programme for potatoes in Bolivia 
evaluated in Bentley et al. (2007), although Bentley and colleagues do not report sufficient 
information on sample distribution to calculate effect sizes. 

There are cases, however, where effect sizes were likely to be dependent. 14 Hiller et al. (2009) 
and Waarts et al. (2012) both reported on the same outcomes for a pilot and up- scaled tea 
IPPM project in Kenya, and Pananurak (2010) and Wu (2010) reported, respectively, short- 
term (one-year follow-up) and medium-term (three-year follow-up) outcomes from the same 
survey of households in China for yields. In these cases we included results for the scaled- up 
project (Waarts et al., 2012) and longer-term follow-up (Wu, 2010) in meta- analysis, 
discussing differences by follow-up time period in the report narratively. 15 In the case of the 
Indonesian National IPM Programme, two papers reported analyses of the same data (Feder 
et al., 2004a; Yamazaki &Resosudarmo, 2008). Given the prominence of these papers in the 
literature, in this case we opted to assess sensitivity of meta- analysis findings to inclusion of 
these studies rather than calculate a synthetic effect size (Appendix G reports results from 
sensitivity analysis excluding these studies; results are not affected) . 

Rejesus et al. (2010) and Huan et al. (1999) estimated impacts on the National IPM 
Programme in Vietnam, collecting data at different points in time. Only a subset of 
observations appeared to be from the same province of Long An. However, we used the 
results from Rej esus on the grounds of preference according to risk of bias, dropping the 
higher-risk-of-bias observations in Huan et al. (1999). Finally, there were two instances of 
studies which reported two measures of the same adoption construct - Yang et al. (2005) 
and Khan et al. (2007), which both reported pesticide use and cost outcomes - in which we 
included only pesticide costs in the pooled meta- analysis of pesticide adoption. 


14 There were a few cases where it was not clear whether studies referred to the same programme. Cole et al. 
(2007) and Mauceri et al. (2007) reported impacts of FFS in the same region of Ecuador, but seemingly for 
different interventions. Yang etal. (2005) and Wu (2010) appeared to estimate impacts on the same programme 
but not the same samples, Yang referring to data in a single province only. It was not dear whether Palis ( 1998) 
and Price (2001) referred to the same intervention conducted in Central Luzon Philippines; however, the studies 
measured different outcomes so there is no ride of violation of the independent effect dze assumption. 

15 Results were not sensitive to the choice of outcome. 
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We have identified effect sizes in the meta- analysis forest plots by author citation and 
country location in the main text. However, we also show the same forest plots identified by 
programme name in Appendix G, so that readers can see which FFS programmes are 
included within each meta- analysis. 

Unit of analysis 

We provided an assessment of the unit of analysis error for the included studies based on 
whether the included studies account for differences in demographic and socioeconomic 
household and village characteristics across individuals in different clusters (Appendix F). 
For those studies that reported moderate or high probability of relevant unit of analysis 
error, corrections were applied to the standard errors and the confidence intervals of the 
effect size using the following formula (Higgins & Green, 2011): 


SEoorrected — SEuncorrected * Vt TKm- 1)*ICC] 

where m is the number of observations per cluster and ICC is the intra-duster correlation 
coeffident, which we assume to be o.05. 16 We used this formula to correct for likdy unit of 
analysis errors in four studies (Bunyatta et al., 2006; Palis, 1998; Rejesus et al., 2010; 
Ricker-Gilbertetal., 2008). The nature of reporting on duster assignment in the original 
studies in the majority of cases meant that we were not able to reach firm conclusions about 
unit of analysis errors. We therefore conducted sensitivity analysis in which additional 
studies assessed as being of "unclear risk" were corrected for unit of analysis errors 
(Appendix G). 

Missing data 

Where possible, we contacted primary study authors to obtain missing information in order 
to fadlitate effed size extraction and risk of bias analysis. Given the large number of studies 
induded in the review, we limited contad to authors of studies which were originally 
assessed as of medium risk of bias (no studies were originally assessed as of low risk of bias) . 

Moderator analyses 

The coding sheet contained in the protocol (Waddington et al., 20 12a) indicated the 
variables we expeded to include in moderator analysis a priori. We identified effed 
moderators according to likely contextual fadors which could affed outcomes, such as 
geographical region, crop type, implementation charaderistics and length of follow-up 
period. Some moderator variables based on intervention and fanner charaderistics were 
determined a posteriori following the review of qualitative literature. We used programme 
name and country to link across quantitative, qualitative and projed portfolio 


16 Unfortunately, to the best of our knowledge, no previous studies have reported ICC for similar interventions 
and outcomes. Campbell et al. (2000) suggested that, overall, the ICC for outcome variables is not bigger than 
0.05 in duster randomised controlled trials and 0.02 is the mean. 
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implementation documentation to enhance the analysis of moderators. We report all 
moderator analyses conducted. 

Methods of synthesis 

We synthesised quantitative data on effectiveness in order to assess the direction and 
magnitude of effects in particular contexts, and mixed methods (quantitative and/ or 
qualitative) data on barriers and enablers. 

Review question (1): Effectiveness synthesis and sensitivity analysis 

We synthesised quantitative information on impacts using inverse- variance weighted 
statistical meta- analysis. We implemented random effects meta- analysis because we can 
reasonably expect effect sizes to differ across studies due to a range of factors including 
contextual variation (e.g. relating to the location, type of crop, beneficiary groups, 
intervention design and implementation process and follow-up period) and study design, 
over and above the effects of chance alone on findings. Random effects meta- analysis 
produces a pooled effect size with greater uncertainty attached to it, in terms of wider 
confidence intervals than a fixed effect model. 

We reported effect sizes on predefined sub-groups of interest, including FFS participants and 
neighbouring farmers potentially exposed to knowledge, to assess the extent of spillovers. 

We also investigated sources of effect heterogeneity according to contextual moderators and 
factors relating to study design. As noted above, there are important issues relating to 
comparability of bivariate and partial effect sizes (Keef & Roberts, 2004; Becker &Wu, 2007; 
Aloe & Thompson, 20 13), due to likely collinearity between treatment variable and 
commonly used covariates such as socioeconomic status and demographic characteristics. 
We therefore conducted sensitivity analyses to examine whether there were any systematic 
associations with effect size magnitude. 

Effect sizes shown using forest plots were synthesised in inverse- variance weighted random 
effects meta- analysis, estimated using Stata software (Stata Corporation, College Station, TX, 
USA) . All response ratio analyses were conducted using the natural logarithm of the effect 
size, with results exponentiated back to the response ratio metric for forest plots and 
discussion. 

Review question (2): Barriers and enablers synthesis 

For the synthesis of evidence relating to question (2), we conducted the synthesis in two 
stages, using the hypothesised programme theory as our overall framework throughout. 

After having completed the detailed coding of all of the included studies we re- reviewed the 
coding and identified descriptive findings which remained dose to the findings in the 
primary studies (following Thomas & Harden, 2008). We summarised the descriptive 
findings across studies, using headings corresponding to the key stages of the FFS 
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programme theory (and the categories used for data extraction) to structure our synthesis. 
The full text of this synthesis is provided in Appendix H. 

The descriptive synthesis provided the basis for a more analytical synthesis. We analysed the 
descriptive findings across studies in detail, and identified themes relating to potential 
barriers and enablers of FFS effectiveness. We then reviewed and compared the original FFS 
programme theory with the emerging evidence on barriers and enablers, and revised the 
theory and assumptions to reflect the findings of our synthesis. 

Integrated synthesis (review questions 1 and 2) 

We used the programme theory (Figure 4) as a framework for integrating the findings from 
the two syntheses (Noyes et al., 20 11) with the aim of providing an integrated narrative 
synthesis along the causal chain addressing the obj ectives of the review. We used an iterative 
approach, in which the intermediate outcomes and assumptions underlying the causal chain 
were further developed as part of the analytical approach. We used programme names to 
link studies included in quantitative, qualitative and project portfolio analyses to facilitate 
moderator analysis. We discuss the implications of the findings for policy, implementation 
and research. 
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4 Results of Effectiveness Synthesis 


4.1 SEARCH RESULTS 


This section reports the results of the synthesis of quantitative studies addressing review 
question ( 1) on the effects of fanner field schools on intermediate and final outcomes for FFS 
participants and non- participating neighbour farmers. We discarded 28, 156 search records 
from the quantitative synthesis at the title or abstract stage, as they did not meet the 
inclusion criteria, being irrelevant to the topic (Figure 8) . Of the 369 potentially relevant full 
text copies which we were able to obtain, 92 evaluation studies were included in the review of 
effects representing 71 FFS interventions conducted in 25 countries. All studies reported in 
English except one in Spanish (Orozco Qrilo et al., 2008). Studies reported on any 
intermediate or final outcomes along the causal chain, some reporting only findings for 
single outcomes, others reporting findings on multiple outcomes. 
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Figure 8 Quantitative synthesis search results 



4.2 STUDY DESCRIPTIVE INFORMATION 


Table 2 summarises descriptive information (reported in full in Appendix E) on the 92 
included impact evaluation studies measuring effectiveness on a range of intermediate and 
final outcomes across the causal chain. Studies were identified from all global regions: 24 in 
East Asia, 25 in South Asia, 11 in Latin America, 1 in Central Asia and 31 in Sub- Saharan 
Africa. Most studies evaluated integrated pest management (I PM) and, particularly in Africa, 
integrated production and pest management (I PPM) farmer field school curricula, although 
a number implemented training on other intensive input management approaches, such as 
integrated crop management (ICM), integrated disease management (IDM) and integrated 
soil management (ISM). 

FFS was provided as part of a multi- component intervention package alongside additional 
intervention components in 11 studies, which included support in procuring inputs and/ or 
marketing produce (Table 2) . Sixteen studies collected data from two treatment groups, the 
FFS participants and the non- participating neighbour farmers living in dose proximity to 
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participants (usually in the same villages) who may benefit from knowledge spillovers via 
contact with FFS farmers. 

The studies collected data on a range of outcomes. Intermediate outcomes were collected for 
knowledge (32 studies) and adoption (59 studies). Agricultural outcomes were frequently 
measured (45 studies), usually in terms of physical measures of production (yields) and 
monetary measures of production which factor in prices (yields value) or take account of 
input costs (net revenue) . A few studies reported other outcome dimensions, such as 
environmental risk factors (6 studies), self-reported health outcomes (4 studies) and 
empowerment (5 studies). For all outcomes, we attempted to calculate effect sizes and 95 per 
cent confidence intervals. However, as explored in Section 4.3 on study quality, in over one- 
third of cases (35 out of 92 FFS programmes) insufficient information was provided on 
sample distribution in order to estimate standard errors and therefore statistical precision of 
the effect sizes; these studies were therefore excluded from statistical meta- analysis. While 
many studies report multiple outcomes, no single study reported data on all intermediate 
and final outcomes along the theory of change. 
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Table 2 Included impact evaluation studies: summary descriptive information by region 


Region 

Intervention 

Countries 

Crop 

Study design 

Risk of bias assessment 


Outcome data collected (effect size calculable) 


Cost 

data 







Knowledge 

Adoption 

Agriculture 

outcome 

Health 

outcome 

Environment 

outcome 

Empowerment 

outcome 

provided 

East Asia & 
the Pacific: 

24 

intervention 

studies 

IPM-FFS (23 studies), 
IPM-FFS+input 
support+marketing 
support (1 study) 

China, 

Indonesia, 

Myanmar, 

Thailand, 

Philippines, 

Vietnam 

Cotton (6 studies), 
rice (13 studies), 
potato/vegetable (5 
studies) 

Longitudinal DID- 
regression (6 studies), 
longitudinal (6 studies), 
cross-section IV 
regression (1 study), 
cross-section 
regression/PS M (3 
studies), 
cross-section (8 
studies) 

Low risk of bias (0 
studies), 

medium (7 studies), 
high (17 studies) 

5 studies (3 
effect sizes) 

16 

studies 
(11 effect 
sizes) 

12 studies 
(8 effect 
sizes) 

1 study 
(0 effect 
sizes) 

1 study (1 
effect size) 

0 studies 

2 studies 

Latin 

America & 
the 

Caribbean: 

11 

intervention 

studies 

IPM-FFS (6 studies), 
IPPM-FFS (1 study), 
"Integrated 

management"-FFS (1 
study), 

FFS+input 

support+marketing (3 
studies) 

Bolivia, 
Ecuador, 
Nicaragua, 
Peru, Mexico 

Cereal (1 study), 
coffee (1 study), 
potato (7 studies), 
vegetable (2 
studies) 

Cross-section IV- 
regression (2 studies), 
cross-section PSM (3 
studies), 

longitudinal (1 study), 
cross-section 
regression (1 study), 
cross-section (5 
studies) 

Low risk of bias (0 
studies), 

medium (3 studies), 
high (9 studies) 

5 studies (4 
effect sizes) 

7 studies 
(6 effect 
sizes) 

6 studies (6 
effect sizes) 

1 study 
(1 effect 
size) 

0 studies 

1 study (1 
effect size) 

3 studies 

Central 
Asia: 1 

study 

Unclear FFS type (1 
study) 

Iran 

Rice (1 study) 

Cross-section (1 study) 

Low risk of bias (0 
studies), 

medium (0 studies), 
high (1 study) 

1 study (1 
effect size) 

1 study (1 

effect 

size) 

1 study (1 
effect size) 

0 studies 

0 studies 

0 studies 

1 study 
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Region Intervention Countries Crop Studydesign Risk of bias assessment Outcome data collected (effect size calculable) Cost 

data 

Knowledge Adoption Agriculture Health Environment Empowerment provided 

outcome outcome outcome outcome 


South Asia: 

IPM-FFS (16 studies), 

Bangladesh, 

Cotton (13 studies), 

Longitudinal DID (2 

Low risk of bias (0 

7 studies (4 

18 

16 studies 

1 study 

25 

ICM-FFS (2 studies), 

India, Nepal, 

rice (8 studies), 

studies), 

studies), 

effect sizes) 

studies 

(9 effect 

(0 effect 

intervention 

IDM-FFS (1 study), 

Pakistan, Sri 

potato/vegetable (2 

cross-section IV- 

medium (2 studies), 


(10 effect 

sizes) 

sizes) 

studies 

ISNM-FFS (1 study), 

Lanka 

studies), 

regression (1 study), 

high (22 studies) 


sizes 




F FS -l-input support (1 


cereal/groundnut (1 

cross-section PSM (2 







study), 


study), 

studies), 







FFS+input 


unclear (1 study) 

cross-section 







support+marketing (2 



regression (1 study), 







studies) 



longitudinal (8 studies), 
cross-section (10 
studies) 






Sub- 

IPM-FFS (8 studies), 

Benin, 

Cocoa (5 studies), 
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Low risk of bias (0 

14 studies 

17 

10 studies 

1 study 

Saharan 

IPPM-FFS (13 studies), 

Cameroon, 

cotton (3 studies), 

(1 study), 

studies), 

(9 effect 

studies (9 

(7 effect 

(1 effect 

Africa: 31 

ICM-FFS (1 study), 

Ethiopia, 

potato/vegetable (14 

longitudinal covariate 

medium (3 studies), 

sizes) 

effect 

sizes) 

size) 

intervention 

ICPM-FFS (2 studies), 

Kenya, 

studies), 

matching (3 studies), 

high (28 studies) 


sizes) 



studies 

IDM-FFS (1 study), 

Ghana, 

cereal/groundnut (5 

cross-section IV- 







ISM (1 study), 

Nigeria, 

studies), 

regression (1 study), 







"Integrated management 

Sudan, 

tea (2 studies), 

cross-section 







practices"^ FS (1 study), 

Tanzania, 

"diverse crops" (1 

regression (6 studies), 







FFS+input supports 

Uganda, 

study), 

cross-section (20 







study), 

FFS+marketing (1 
study), 

FFS+input 

support+marketing (2 
studies) 

Zimbabwe 

unclear (1 study) 

studies) 







4 studies (2 0 studies 

effect sizes) 


> studies 


1 study (1 
effect size) 


4 studies (0 
effect sizes) 


1 study 


Notes: FFS type: IPM=integrated pest management; IPPM=integrated production and pest management; ICM=integrated crop management; ICPM=integrated crop and pest management; IDM=integrated 
disease management; ISM=integrated soil management; ISNM=integrated soil nutrient management. Studydesign: DID ^difference- in- differences; IV=instmmental variables; PSM=propensity score 
matching. 

Source: based on data reported in Appendix E and Appendix F. 
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The 92 included studies covered 71 distinct FFS programmes (see Section 3.3 for a 
discussion of dependent effect sizes). The activities forming the components of included FFS 
interventions are summarised in Figure 9. 17 In most cases detailed information on the 
intervention activities was not recorded dearly. For example, although one- third of 
programmes did mention training- of- trainers, almost all of these did not clarify the training 
used and it was clear in only three cases that the training approach was inadequate. 

The extent to which FFS adapted to different contexts are based on the same process, both in 
the design and implementation of the programmes, varies (Davis et al., 20 12) . There appear 
to be two main types of FFS programme implemented in practice. About half of FFS impact 
evaluations were implemented by (or in partnership with) the FAO based on a partidpatory 
fadlitation approach. Several studies appeared to implement a top-down transfer of 
technology approach based on 'lecturing" (Todo &Takahashi, 2011; Yanget al., 2008). 
However, in a large number of cases it was simply undear how the intervention was 
designed or implemented. Thus, while half dearly indicated that they used partidpatory 
fadlitation, it was not dear in most of the remaining half which teaching pedagogy was used. 
Schools were usually preceded by development of spedfic curricula relevant to the context in 
which schools were being implemented. Farmer group formation and fadlitator training for 
extension workers or other field school staff were usually also carried out prior to FFS 
training. In some cases training of fadlitators appeared to be limited to attendance at a 
workshop (e.g. Erbaugh et al., 2010), rather than season- long theoretical and practical 
training, while in other cases fadlitator training was judged unnecessary given existence of 
previously trained fadlitators as, for example, in Kenya and Tanzania (Friis- Hansen & 
Duveskog, 2012). 

Most schools hdd weekly season-long sessions, although in a few cases arable crop meetings 
were held fortnightly rather than each week (e.g. Erbaugh et al., 20 10) . In the case of 
permanent crops, such as tea and cocoa, meetings were often held fortnightly over the course 
of several months (Hiller et al., 2009; David &Asamoah, 2011) or years (e.g. Endalew, 

2009). The activities implemented by the schools appeared to indude agro- ecosystem 
analysis and "farmer practice" control plots in 50 per cent of cases each, while group 
dynamics, spedal topics and exchange visits were less common or were not reported. 

The extent to which diffusion of IPM practices is assumed to occur informally through FFS 
graduates' sodal networks, rather than needing to be formally encouraged through training- 
of- trainer programmes for alumni, varies from programme to programme. Follow-up 
activities to foster dissemination such as fidd days, training-of- trainer programmes for 
farmer fadlitators and other means of community institutionalisation were only reported in 
a minority of cases (see also Appendix A) . 


17 Data were coded on intervention design not implementation, which was usually unavailable. 
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Figure 9 Characteristics of programmes included in review of effects 
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Notes: data from included studies and portfolio project documentation (Appendix A). Due to under- 
reporting it is likely that the figures underestimate characteristics of programmes implemented. 
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4.3 ASSESSMENT OF STUDY VALIDITY 


Risk of bias assessment 

Designing impact evaluations of agricultural programmes is complicated by the wide range 
of factors that influence agricultural outcomes and by biases caused by self- selection of 
individuals and communities into programmes. Thus, differences in outcomes between 
participants and non- participants might result from pre-existing differences rather than 
being attributable to the programme under evaluation (Romani, 2003) . 

These problems arise in attempts to attribute the impact of farmer field school programmes 
on agricultural (or other 'final') outcomes. For instance, where "the selection of participants 
into the training is done with strong community involvement through its established 
leadership and existing social structures" (Feder et al., 20 10, p. 10), certain farmers, such as 
community leaders or those of relatively high socioeconomic status, may be more likely to 
benefit from the intervention than others. Other programmes, particularly those in Africa, 
aimed to target disadvantaged farmers such as women (see Appendix A). In addition, pilot 
programmes may be explicitly placed where they are likely to have the greatest impact. And 
in the case of I PM field schools, explicit programme objectives were that benefits spill over 
from FFS participants to non- participating neighbour farmers. In other words, the unit of 
assessment should be at the community rather than the household level. 

In the case of evaluating impacts on agricultural outcomes, such as yields and incomes, the 
likelihood of serious confounding, particularly by weather and market prices, means that 
appropriate methods of addressing the attribution challenge necessarily involve equivalent 
or matched comparison groups. One might argue that impact evaluations drawing on less 
causally rigorous standards of evidence would be appropriate for intermediate outcomes of 
interest, such as knowledge or adoption of new technologies, particularly where it is unlikely 
that beneficiaries would otherwise know about the technologies, especially in the case of 
complex messages. 

In the case of adoption, farmer behaviour is influenced by a range of factors, including policy 
changes, which are likely to confound impact estimates. Removal of subsidies and banning of 
certain pesticides, as happened in Indonesia in the late 1980s (Braun & Duveskog, 2008), 
are examples of factors that would likely influence farmers' pesticide (dis-) adoption 
behaviour. In such contexts, a "before versus after" evaluation (pre-test post- test with no 
comparison group) would not enable researchers to attribute changes to any specific 
extension interventions. Similarly, farmers might gain knowledge from several places, 
including public information campaigns, other farmers and other extension interventions. 
For instance, in Vietnam a "no early spray" media campaign was run at the same time as 
FFS, and a study comparing neighbouring farmers who were exposed to the media campaign 
with those also attending FFS and a comparison group not exposed to either intervention 
found that beliefs about insecticide spraying changed among those exposed to both FFS and 
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the media campaign and that the two interventions appeared complementary (Huan et al., 
1999) . In this case, a simple "before versus after" comparison would have overestimated the 
impact of the FFS programme on knowledge of simple practices (although not more complex 
knowledge) . 

During the period in which we conducted the analysis, we were unable to identify any 
completed experimental studies based on randomised assignment . m Despite the threats to 
validity being well known, and the feasibility of conducting cluster- randomised controlled 
trials (RCTs) for FFS interventions, the majority of FFS impact evaluations use designs of 
questionable internal and statistical conclusion validity, and therefore have high risk of bias 
in attributing final outcomes to the intervention. Treatment effects were usually estimated 
relative to a non-intervention comparison population, using contemporaneous cross-section 
data, although there are 30 studies of longitudinal design utilising panel data or group 
comparison (Table 2). 

The summary risk of bias report by categories of selection bias and confounding, spillovers, 
reporting biases, and other sources of bias is provided in Figure 10 (individual assessment 
for each study is reported in Appendix F) . 19 Of the included studies, 68 were classified as of 
high risk of bias, many of which were retrospective evaluations without baseline 
measurement, and, in a large number of cases, no attempt was made to match participant 
and non- participant covariates or control for such covariates in analysis. Twenty- four studies 
used more rigorous quasi- experimental approaches, including multivariate propensity score 
matching (e.g. Godtland et al., 2004) and covariate matching (Davis et al., 2012), 
multivariate instrumental variables regression (e.g. Ricker- Gilbert et al., 2008) and, where 
baseline and endline data were collected, multivariate difference- in- differences (or fixed 
effects) panel data regression (e.g. Feder et al., 2004; Wu, 2010). These studies adjusted for 
different covariates in the outcome equations, usually involving measures of household 
socioeconomic status and farmer demographics. However, our assessment also found many 
of these studies to be at serious risk of bias, usually due to problems of confounding (e.g. lack 
of tests for covariate balance or adjustment for unbalanced covariates in outcome equations, 
or inappropriate instrumental variables employed). Only 15 intervention studies were 
considered sufficiently rigorous to qualify as of medium risk of bias (Table 2), all of which 
used multivariate estimation methods, although none of these studies indicated that they 
used blinding either of outcome assessors or of data analysts. We were not able to identify 
any studies which could be described as having a low risk of bias in attributing outcomes to 
the intervention. 

Another common weakness of the FFS impact evaluations was that they mostly relied on 
small samples. The median sample size is 185 farmers and samples range from 21 to 960. 


18 We are aware of two on-going experimental studies (randomised assignment) of FFS in China (Rodriguez et al. 

: http://www.3ieimpact.org/en/evidence/impact-evaluations/details/206/ ) and the Philippines (Masset & Haddad, see 
http:/ / www.ids.ac.uk/ files/ dmfile/ CDI_ EMasset_ ParFARM_ 06iunl3.pdf ). 

19 Full details of the critical appraisal are available on request from the authors. 
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Often only a handful of primary sampling units are covered (median of 8 dusters or villages 
across treatment groups) . Statistical power is thus limited. 

Most studies appeared to assess pilot or small-scale projects rather than scaled- up 
programmes. There are two prominent exceptions: the Indonesia National IPM Programme 
(evaluated by Feder et al., 2004 and Yamazaki &Resosudarmo, 2008) and the Vietnam 
National IPM Programme evaluated by Rejesus et al. (2009). However, both have 
comparably small sample sizes (in the case of Feder et al., n=320; in the case of Rejesus et 
al., n=43 and there is only one commune primary sampling unit) . 

Figure 10 Summary of risk of bias appraisal for effectiveness studies 
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Overall risk of bias assessment 


1) Selection bias & confounding 


2) Spillovers & contamination 


3) Outcome reporting bias 


4) Analysis reporting bias 


5) Other sources of bias 



■ Low risk of bias ■ Medium risk of bias ■ High risk of bias 


Source: authors based on Appendix F. 

The availability of details on the intervention and comparison groups varied and was often 
limited. Most studies provided little information on the comparison group intervention apart 
from stating that the comparison farmers did not receive FFS training, or that there was no 
FFS intervention in the village; in a few cases, the studies indicated where comparisons had 
access to other sources of information about intensive input management approaches (e.g. 
Davis et al., 20 12, in Uganda) . We assessed whether there were any effects from these 
potentially contaminated comparisons in heterogeneity analysis by risk of bias status. In 
many cases, the comparison group was selected from among farmers living in the same 
village as the FFS (e.g. Carlberg et al., 2012; Dankyi et al., 2005; Islam et al., 2006; Kabir & 
Uphoff, 2007; Mutandwa&Mpangwa2004; Pouchepparadjouetal., 2005; Price, 2001; Rao 
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et al., 2012; Wandji et al. 2007), or induded both non-FFS farmers from different locations 
and neighbouring farmers living in the same location (e.g. Rejesus et al., 2010; Ricker- 
Gilbert et al., 2008). 

Given the voluminous impact evaluation literature on farmer field schools, our estimation 
has indicated that there are a number of high- risk- of- bias studies which are of very limited 
validity in attributing causal effects, particularly for final outcomes such as agricultural 
yields and revenue. We therefore interpreted the results of the synthesis with caution, 
reporting both pooled and stratified analyses based on risk of bias categories in meta- 
analysis, and making implications for policy in terms of effect sizes based only on those 15 
studies regarded as being of medium risk of bias. Table 3 provides more detailed information 
on studies that we consider appropriate for informing policy due to their risk of bias 
assessment. 

Of the full sample of studies, we estimated unit of analysis errors were "highly probable" in 
four studies, for which corrections were applied to all effect size standard errors. We also 
estimated unit of analysis errors to be "unclear" but possible in 47 studies, largely due to lack 
of clarity in responding on cluster sampling (Appendix F 20 ) . We therefore also presented 
sensitivity analyses for additional studies which were assessed as of "undear" unit of analysis 
error (Appendix G), the results being broadly similar to the main findings for medium risk of 
bias studies, although the pooled effect sizes for high risk of bias studies tended to be of 
smaller magnitude for FFS partidpants. 


20 The assessment was reported as not applicable (N/ A) mainly in those cases where we were unable to compute 
standard errors, as well as for a smaller number of studies which did not use duster design. 
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Table 3 Detailed descriptive information on study design, method of analysis and sample size (high- risk-of- bias studies excluded) 


Study 

Location 

Description of FFS 
intervention 

Study arms 

Study design; method of 
attribution 

Follow-up 

time 

period 1 

Sample size 2 
(num clusters) 

Unit of analysis 
error assessment 

Ali and Sharif 
(2012) 

Pakistan 

FFS providing training on IP M, 
cotton (programme name 
unclear) 

(1) FFS-IPM (2) non-FFS 
comparison (intervention not 
stated) 

Multivariate: cross- 
sectional data; propensity 
score matching 

Not stated 

325 (num village 
clusters not stated 
but data collected 
from 7 districts) 

Possibility of relevant 
unit of analysis error: 
authors include district 
dummies but not 
village dummies 

Cavatassi etal. 
(2011) 

Ecuador 

Plataformas de Concertacion 
NGO-implemented 
programme including training 
on IPM + farmer cooperatives 
organisation +new seeds, 
potato farmers 

(1) FFS-IPM + farmer 
organisation and new seeds (2) 
non-FFS-plus comparison 
(intervention not stated) 

Multivariate: cross- 
sectional data; propensity 
weighted least squares 
(weights calculated from 
propensity score matching 
regression) 

1-4 years 

835 of which 292 
FFS, 543 non-FFS 
(35 communities, 
18 FFS, 17 non- 
FFS) 

Low probability of 
relevant unit of 
analysis error: the 
authors include 
controls for location- 
specific effects 
accounting for 
clustering 

Davis etal. (2012) 

Kenya 

IFAD-FAO East Africa 
Programme forIPPM, maize, 
bean and vegetable farmers 

(1) FFS-IPPM (2) non-FFS 
comparison (intervention not 
stated) 

Multivariate: controlled 
before versus after 
(individual panel) data; 
covariate matching, 
bivariate difference-in- 
differences 

2 years 

398 of which 281 
FFS and 117 non- 
FFS (20 villages of 
which 10 FFS) 

Low probability of 
relevant unit of 
analysis error: authors 
include a village fixed 
effect 

Davis etal. (2012) 

Tanzania 

IFAD-FAO East Africa 
Programme forIPPM, maize, 
bean and vegetable farmers 

(1) FFS-IPPM (2) non-FFS 
comparison (intervention not 
stated) 

Multivariate: controlled 
before versus after 
(individual panel) data; 
covariate matching, 
bivariate difference-in- 
differences 

2 years 

379 of which 272 
FFS and 107 non- 
FFS (20 villages of 
which 10 FFS) 

Low probability of 
relevant unit of 
analysis error: authors 
include a village fixed 
effect 

Federetal. (2004) 

Indonesia 

National IPM Programme, rice 
farmers 

(1) FFS-IPM (2) FFS neighbours 
(3) non-FFS comparison 
(traditional public extension 
services) 

Multivariate: controlled 
before versus after 
(individual panel) data; 
multivariate difference-in- 
differences estimation 

3-5 years 

320 of which 112 
FFS, 156 FFS 
neighbours, 52 
non-FFS (26 
villages of which 

Low probability of 
relevant unit of 
analysis error: authors 
use DID which 
accounts for 


Study 

Location 

Description of FFS 
intervention 

Study arms 

Study design; method of 
attribution 

Follow-up 

time 

period 1 

Sample size 2 
(num clusters) 

Unit of analysis 
error assessment 







21 FFS and 5 non- 
FFS) 

clustering in the 
treatment allocation 

Godtland etal. 
(2004) 

Peru 

NGO (CARE) FFS providing 
training on IP M , potato 
farmers 

(1) FFS-IPM (+ Andino) (2) 
traditional public extension 
services CARE Andino 
programme 

Multivariate: cross- 
sectional data; propensity 
score matching 

<1 year 

90 (45 FFS 
farmers, 45 
control) (10 
villages: 4 FFS, 6 
Andino) 

Possibility of relevant 
unit of analysis error: 
authors do not match 
on village-level 
characteristics 

Pananurak (2010) 

China 

FAO-EU IPM Programme for 
cotton farmers 

(1) FFS-IPM (2) FFS neighbours 
(3) non-FFS comparison 

Multivariate: controlled 
before versus after 
(individual panel) data; 
multivariate difference-in- 
differences estimation 

2 years 

535 of which 177 
FFS, 178 FFS 
neighbours, 180 
non-FFS (18 
villages, 9 FFS, 9 
non-FFS) 

Low probability of 
relevant unit of 
analysis error: authors 
use DID 

Pananurak (2010) 

Pakistan 

FAO-EU IPM Programme for 
cotton farmers 

(1) FFS-IPM (2) FFS neighbours 
(3) non-FFS comparison 

Multivariate: controlled 
before versus after 
(individual panel) data; 
multivariate difference-in- 
differences estimation 

2 years 

190 of which 78 
FFS, 59 FFS 
neighbours, 53 
non-FFS (8 
villages, 4 FFS, 4 
non-FFS) 

Low probability of 
relevant unit of 
analysis error: authors 
use DID 

P raneetvatakul 
and Waibel (2006) 

Thailand 

Government of Thailand pilot 
IPM-FFS programme, rice 
farmers 

(1) FFS-IPM (2) FFS neighbours 
(3) non-FFS comparison 
(intervention not stated) 

Multivariate: controlled 
before versus after 
(individual panel) data; 
multivariate difference-in- 
differences estimation 

3 years (3 
period 
model 
used) 

188 for 3 period 
model (241 for 2 
period model, of 
which 107 FFS, 58 
FFS neighbours 
and 76 non-FFS) 
(10 villages of 
which 5 FFS) 

Low probability of 
relevant unit of 
analysis error: authors 
use DID 

Rejesus etal. 
(2010) 3 

Vietnam 

Vietnam National IPM 
Programme, rice farmers 

(1) FFS-IPM (2) FFS neighbours 
(3) non-FFS comparison 
(intervention not stated but no 
IPM "no early spray" media 
campaign) 

Multivariate: controlled 
before versus after 
(individual panel) data; 
multivariate difference-in- 
differences estimation 

7 years 

44 of which 11 
FFS, 33 
neighbours (1 
commune) 

High probability of 
relevant unit of 
analysis error: farmers 
from only one 
commune are 
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Study 

Location 

Description of FFS 
intervention 

Study arms 

Study design; method of 
attribution 

Follow-up 

time 

period 1 

Sample size 2 
(num clusters) 

Unit of analysis 
error assessment 








included in the 
analysis 

Todo and 
Takahashi (2011) 

Ethiopia 

FFS as part of a wider 
programme by Oromia Forest 
and Wildlife Enterprise and 
J ICA to prevent deforestation 
and promote reforestation in 
the Belete-G era Regional 
Forest Priority Area, cabbage, 
onion, carrot and beet farmers. 

(1) FFS with a wide curriculum 
including learning of new 
agricultural technologies, such 
as farm management, seedbed 
preparation, proper spacing, 
new varieties, and sowing 
methods. Also the FFS teaches 
how to nurse trees and promote 
reforestation. Also, it seems that 
participants may have support to 
obtain certification for coffee. (2) 
non-FFS comparison. 

Multivariate: controlled 
before versus after 
(individual panel) data; 
PSM and difference-in- 
difference estimation 

1-2 years 

269 of which 55 
FFS and 214 non- 
FFS (8 FFS sub- 
villages and 16 
non-FFS sub- 
villages) 

Possibility of relevant 
unit of analysis error: 
it is not clear whether 
clustering has been 
taken into account in 
the regression 
analysis 

Van Rijn (2010) 

Peru 

FFS + cooperative building + 
certification, coffee farmers 

(1) FFS + cooperative + 
certification (2) non FFS-plus 
comparison 

Multivariate: cross- 
sectional data: propensity- 
score matching 

3-4 years 

200 of which 93 
FFS and 107 non- 
FFS (18 villages, 9 
FFS and 9 non- 
FFS villages) 

Low probability of 
relevant unit of 
analysis error: authors 
match on village 
characteristics 
accounting for 
clustering in the 
treatment allocation 

Wu (2010) 

China 

FFS FAO-EU IPM Programme 
for cotton farmers in Asia 

(1) FFS-IPM (2) FFS neighbours 
(3) non-FFS comparison 
(intervention not stated) 

Multivariate: controlled 
before versus after 
(individual panel) data; 
multivariate difference-in- 
differences estimation 

2 years 

480 of which 155 
FFS, 158 FFS 
neighbours, 167 
non-FFS 
comparison (18 
villages, 9 FFS, 9 
non-FFS) 

Low probability of 
relevant unit of 
analysis error: authors 
use DID 

Yamazaki and 
Resosudarmo 
(2008) 

Indonesia 

National IPM Programme, , 
rice farmers 

(1) FFS-IPM (2) FFS neighbours 
(3) non-FFS comparison 
(traditional public extension 
services) 

Multivariate: controlled 
before versus after 
(individual panel) data; 
multivariate difference-in- 

4 years 

320 of which 112 
FFS, 156 FFS 
neighbours, 52 
non-FFS (26 

Low probability of 
relevant unit of 
analysis error: authors 
use panel data 
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Study 

Location 

Description of FFS 
intervention 

Study arms 

Study design; method of 
attribution 

Follow-up 

time 

period 1 

Sample size 2 
(num clusters) 

Unit of analysis 
error assessment 





differences estimation 


villages of which 
21 FFS and 5 non- 
FFS as per Feder 
etal.) 

models 

Yorobe etal. 
(2011) 

Philippines 

IPM-CRSP South-East Asia 
Regional Programme of 
USAID, onion farmers 

(1) FFS-IPM (2) non-FFS 
comparison 

Multivariate: cross- 
sectional data; two-step 
approach including 
instrumental variables and 
Heckman procedures 

2 years 

200 of which 69 
FFS and 131 non- 
FFS (8 barangays, 
4 FFS and 4 non- 
FFS) 

Possibility of relevant 
unit of analysis error: 
authors include 
regional dummies but 
not barangay 
dummies 


Notes : 1 Average years from start of implementation of intervention to endline survey . 2 Total number of farmers in treatment and comparison 
groups. 3 Comparison group in FFS effect estimates is FFS neighbour group. 
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External validity 

Of the 92 impact evaluations covered by the quantitative review of effects, 42 provided 
descriptive information about FFS fanners (rather than characteristics of the fanners in the 
village as a whole) together with information on sampling methodology. Through this 
information we were able to say something about external validity (generalisability) for 
farmers who underwent FFS training in programmes covered by the effectiveness review, as 
well as for farmers in low- and middle- income countries more generally. 

Table 4 provides an overview of the sampling approach used by these studies, most sampling 
either the entire population of FFS farmers or a random sample. 


Table 4 Sampling approach used in studies providing data on FFS farmer characteristics 


Studies 

Sampling of field 
schools 

Sampling of 
farmers 

Haiyang, 2002 (China); Khan etal., 2007 (Pakistan); Lama etal., 2003 (Nepal); 
Palis, 1998 (Philippines); Rola etal., 2002 (Philippines); Van Rijn, 2010 (Peru) 

All schools 

All farmers 

Davis etal., 2012 (Kenya, Tanzania, Uganda); Friis-Hansen and Duveskog, 2012 
(Kenya, Tanzania, Uganda); Erbaugh etal., 2010 (Uganda); Islam etal., 2006 
(Bangladesh); Kelemework, 2005 (Ethiopia); Naik et al., 2010 (India); Pananurak, 
2010 (India); Pananurak, 2010 (Pakistan); Praneetvatakul and Waibel, 2006 
(Thailand); Rejesus etal., 2010 (Vietnam) 

Random 

Random 

Rao et al., 2012 (India); Zuger, 2004 (Peru) 

Random 

Unclear 

Chi etal., 1999 (Vietnam); David, 2007 (Cameroon); Waarts etal., 2012 (Kenya) 

Unclear 

Random 

Carlberg etal., 2012 (Ghana); Cavatassi etal., 2011 (Ecuador); Friis-Hansen etal., 
2004 (Uganda); Ricker-G ilbert et al., 2008 (Bangladesh); Tripp etal., 2005 (Sri 
Lanka); Yang etal., 2008 (China); Yorobe etal., 2011 (Philippines) 

Purposive 

Random 

Endalew, 2009 (Ethiopia); Khalid, n.d. (Sudan) 

Purposive 

Purposive 

DeJ ageretal., 2009 (Kenya); Godtland etal., 2004 (Peru); Olanya etal., 2010 
(Uganda); Van den Berg and Amarasinghe, 2003 (Sri Lanka); Yang etal., 2005 
(China) 

Purposive 

Unclear 

Feder et al., 2004 (Indonesia); Yanazaki and Resosudarmo, 2008 (Indonesia); 
Todo and Takahashi, 2011 (Ethiopia) 

Unclear 

Unclear 


Six studies collected data on the entire population of FFS farmers, while a further 26 
employed random selection either of field schools or villages or of participants, or both. Of 
those remaining, five reported that they had selected the FFS purposively, but were unclear 
how FFS farmers were sampled or whether the entire population of FFS farmers was 
included. Three reported that FFS farmers had been selected purposively with the goal of 
including either a balanced sample of male and female farmers (Endalew, 2009) or to ensure 
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that full-time fanners that had attended FFS sessions regularly were included (Khalid, n.d.) . 
The selection criteria of farmers in Todo and Takahashi (2011) was unclear, although the 
authors stated farmers were selected "as randomly as possible" (p. 11), and a final study drew 
on random sample household survey data in Indonesia, although it is not dear to what 
extent the sampling frame was representative of the population of field school farmers 
(Feder et al., 2004). 

The induded studies provided information on the characteristics (e.g. age, sex, education 
levds, size of landholdings) of partidpating and non-partidpating neighbour farmers for 48 
of the 71 FFS interventions. As shown in Figure 11, the results suggested FFS farmers are on 
average more likely to be male, over 40 years of age, to have access to at least 1 hectare of 
land, and have completed primary education (attended seven years of education). However, 
analysis across the different programmes indicated that there are FFS interventions that 
involve a high degree of partidpation of poorer, younger and less- educated farmers, as well 
as those that indude large numbers of educated and better-off farmers in terms of 
landholdings. FFS programmes employ a variety of targeting criteria to reach different 
groups, reflecting the often contrasting overarching aims and obj ectives of different 
programmes (Phillips et al., 2014). Some programmes included a majority of partidpants 
that can be considered the most capable farmers in a community - those who are well 
educated, well connected and organised, and have access to resources - with the explidt goal 
of ensuring that partidpants were those most capable of capitalising on the training. Other 
programmes explidtly set out to be pro- poor or indusive of a range of different groups, 
including women and youth. Indeed, years of education varies between five and nine years 
(based on 20 observations), while the size of landholdings ranged between less than half a 
hectare and around nine hectares (based on 21 observations), reflecting the studies drawn 
from regions with different population densities. The average partidpation of women was 32 
per cent, although the proportion of female partidpants ranged from as low as 8 per cent to 
as high as 72 per cent (based on 17 observations) (Figure 11). 

In summary, while there are prominent examples where it is not dear how representative 
study partidpants are of the broader FFS programme (Feder et al., 2004; Yamazaki & 
Resosudarmo, 2009), in the majority of cases study partidpants appear to have been 
selected at random. On the other hand, the farmer groups of which FFS partidpants are 
representative vary according to the FFS project; some appear to involve more educated 
farmers, while others explidtly target women and marginalised groups. 
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Figure 11 Kernel density histograms showing farmer characteristics 
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File-drawer effects and publication bias assessment 

A large number of studies reported data on intermediate and final outcomes. According to 
the risk of bias criteria reported in Figure 10, we did not consider studies with outcome 
reporting biases to outweigh those without. However, it is particularly difficult to assess file- 
drawer effects in retrospective studies where results- related choices can be made in selecting 
outcomes to report. No single study collected data on all outcomes along the theory of 
change. Some 27 studies did not provide information on yields or other agriculture outcomes 
despite collecting data on knowledge or adoption, which might suggest under-reporting of 
the former. But agriculture outcome data are notoriously more difficult to collect (yields, for 
example, requiring accurate information on both weight or value of produce and land size) . 
These are usually study designs where causal inferences on final outcomes would in any case 
be challenged. 

Nevertheless, as in many other areas of social science research, we believe there were 
potentially severe problems of file- drawer effects in the FFS impact evaluation literature. 
Studies used a wide range of different outcome definitions to measure similar constructs 
(Table 1 and Appendix E). 21 Moreover, sufficient information was only available to calculate 
effect sizes in 95 out of 151 cases in total, while insufficient information was provided to be 
able to estimate standard errors, and therefore statistical precision of the effect sizes, in one- 
third of cases (Appendix E), indicating serious problems in under-reporting. 22 

We conducted statistical analyses to assess likelihood of publication bias, including funnel 
graphs and meta- regression analysis incorporating the standard error of the effect size as an 
explanatory variable (Egger et al., 1997) . Funnel graphs for those outcomes with ten or more 
observations (Figure 12) suggest under-reporting of small sample studies for knowledge, 
adoption of practices and yields. However, visual inspection is unreliable and, in any case, 
funnel plot asymmetry may be due to other factors such as methodological quality (smaller 
studies with lower quality may have exaggerated effect sizes), sources of artefactual variation 
such as heterogeneity in outcome measurement, and true heterogeneity due to intervention 
characteristics (Sterne & Egger, 2001). 

Meta- regression analysis enabled the inclusion of additional covariates measuring these 
factors, including the summary risk of bias assessment, differences in outcome measurement 
and contextual factors such as intervention type, crops and region (Table 55 specifications 
2) . The meta- regressions suggested small study effects may be present using bivariate 
Eggeris tests (Table 5 specifications 1). However, the small study effects are robust to 
inclusion of additional covariates in the case of yields only, providing evidence suggestive of 

21 This was not necessarily due to file-drawer effects. For example, in the case of Davis et al. (20 12), the authors 
used monetary value of yields rather than the more conventional yields weight used in most other studies, which 
the authors noted is due to the extent of multi- cropping (farmers growing multiple crops on the same plot and 
growing season). 

22 In addition, seven studies did not provide sufficient information to calculate effect sizes for relevant 
comparison groups (Achonga et al., 2011; Bentley et al., 2007; Mangan &Mangan, 1998; Maumbe & Swinton, 
2003; Olanya et al., 2010; Pouchepparadjou et al., 2005; Yanget al., 2008). 
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publication bias for that outcome variable. The significantly positive coefficient estimate on 
the dummy variable indicating whether the study was published in a peer- reviewed j oumal 
also provides further support for publication bias in favour of studies with larger effects for 
yields outcomes . 23 


Figure 12 Funnel graphs by outcome 
Knowledge 


Funnel plot with pseudo 95% confidence limits 



-.5 0 .5 1 1.5 2 

smdcox 


Pesticide use 


Adoption of other practices 


Funnel plot with pseudo 95% confidence limits 



"5 d 0 1 2 3" 

Pooled response ratio 


Funnel plot with pseudo 95% confidence limits 



-1 -.5 0 .5 1 1.5 

smdcox 


Yields 


Net revenue 


Funnel plot with pseudo 95% confidence limits 



10 _ 

-1 -.5 0 .5 1 

Pooled response ratio 


Funnel plot with pseudo 95% confidence limits 



- 2-10123 
Pooled response ratio 


23 Results were not sensitive to inclusion of additional covariates, including categorical variables measuring 
outcome, global region and crop type (findings not reported) . Results were not sensitive to exclusion of high risk 
of bias studies from the analysis: exponentiated coefficient on log standard error foryields=5.00, t-statistic=2.33, 
P-value=0.049. 
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Table 5 Meta- regression analysis of small study effects (Egger's test) for FFS participants 


Outcome variable Knowledge (SMD) Pesticide use (RR ) 1 Adoption (SMD) Yield (RR ) 1 Revenue (RR ) 1 



a) 

(2) 

(i) 

(2) 

(i) 

(2) 

(i) 

(2) 

(i) 

(2) 

Standard error of effect 
size (Egger's test) 1 

2.16 

(2.58)** 

0.18 

(0.13) 

0.53 

(0.89) 

0.52 

(0.93) 

1.15 

(0.85) 

0.73 

(0.37) 

4.58 

(2.17)** 

6.02 

(2.60)** 

1.10 

(0.10) 

0.88 

(0.14) 

l=medium risk of bias 
study (dummy 
variable) 


0.023 

(0.12) 


1.26 

(1.07) 


-0.33 

(0.77) 


0.86 

(1.47) 


0.90 

(0.38) 

1=P ublished in journal 
(dummy variable) 








1.21 

(1.79)* 



1=C ox-transformed 
SMD (dummy 
variable) 


-0.49 

(1.66) 




-0.01 

(0.03) 





1= T F S -plus" 
intervention 
(dummy variable) 










1.94 

(2.43)** 

Constant 

0.20 

(1.42) 

0.69 

(2.29)** 

0.77 

(1.68) 

0.70 

(2.01)* 

0.40 

(1.62) 

0.53 

(1.12) 

1.10 

(1.01) 

1.01 

(0.08) 

1.58 

(2.30)** 

1.40 

(1.78) 

Tau-sq 

0.08 

0.06 

0.20 

0.19 

0.33 

0.38 

0.06 

0.06 

0.20 

0.14 

l-squared res 

92% 

90% 

94% 

94% 

99.5% 

99.6% 

89% 

89% 

93% 

89% 

Adjusted R-sq 

36% 

50% 

-1% 

4% 

-1% 

-14% 

13% 

24% 

3.2% 

30% 

Sample size 

18 

18 

22 

22 

15 

15 

29 

29 

14 

14 


Note: 1 Meta- regressions based on response ratio (RR) effect sizes are estimated using logged RR and logged 
standard errors; exponentiated coefficients reported. Absolute value of t- statistic reported in parentheses; * **, 
*** indicates coefficient significant at <10% <5% and <1% levels. Models estimated using inverse- variance 
weighted random effects analysis. 
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4.4 META-ANALYSIS RESULTS 


This section reports the results of meta- analysis for effectiveness of farmer field school 
training for participants and non- participant neighbour fanners potentially experiencing 
diffusion effects through their interactions with beneficiaries. We present results using the 
theory of change (Figure 4), for intermediate outcomes (knowledge and adoption) and final 
outcomes (agriculture, health, environment and empowerment). The meta- analysis is 
reported by outcomes rather than study. No single study reported all outcomes along the 
causal chain, although we have attempted to fink outcomes for individual studies in order to 
explain heterogeneity in findings. 

The FFS were carried out using different intervention designs, for participants of different 
demographic and socioeconomic backgrounds, by different implementing bodies, and using 
different impact evaluation estimation methods, which we would reasonably expect to have 
an impact on effect size over and above sampling error. We therefore used random effects 
meta-analysis to synthesise the findings. We present effects for I PM/ IPPM FFS and other 
FFS curricula, and also present results for diffusion to neighbours of I PM- FFS where these 
were reported in the original studies. We synthesised findings from bivariate and partial 
effect sizes, which are only strictly comparable under the assumption of constant slopes 
(there is no collinearity between treatment effect and covariates; see Appendix D) . We 
therefore conducted and report extensive sensitivity analysis by evaluation design features, 
including whether effect sizes were estimated from bivariate or multivariate analyses. All 
medium- risk-of- bias studies, on which policy implications regarding effects are based, used 
multivariate estimation strategies producing partial effect sizes. However, not all used the 
same variable constructs and are therefore not strictly comparable where collinearity exists. 
Finally, we analysed heterogeneity according to context and implementation factors using 
moderator analysis and meta- regression. 

Capacity building and knowledge dissemination (intermediate outcomes) 

Effect sizes for knowledge are measured in terms of standardised mean difference (SMD), 
indicating the change in knowledge scores among FFS (or neighbour) farmers over the non- 
FFS comparison, measured in terms of standard deviations of the outcome variable. SMD 
scores are interpreted as the number of standard deviation changes in the outcome. 

The studies presented a range of knowledge variables (Appendix E), the majority comparing 
beneficiaries to non- beneficiaries according to average points on a test score (or a knowledge 
index as in the case of Huan et al., 1999) across multiple categories of crop management. 
Evidence suggests farmer field schools lead to significant capacity building in farming 
techniques among FFS participants. 

The meta- analysis of standardised mean differences (Figure 13) indicates positive effects of 
FFS training on knowledge of beneficial farming practices for both I PM/ IPPM curricula 
(SMD=0.56, 95% confidence interval (CI)=0.31, 0.80; Q=114, Tau-sq=0.12, 1-sq=92%; 10 
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observations) and curricula (SMD=0.38, 95% Cl =0.22, 0.54; Q=131, Tau-sq=0.04, 1- 
sq=95%; 8 observations), in comparison with untrained farmers. 24 While all observed 
studies showed positive impacts on participant knowledge for FFS farmers, there is 
significant variation in magnitude of impacts ( as indicated by the value of I- squared) . 

Figure 13 Knowledge outcomes for FFS participants and neighbours versus non- FFS 
comparison 


Study 

ID SMD (95% Cl) 


IPM/IPPM FFS 
Godtland et at, 2004 (Peru) 

Rejesus et at, 2010 (Vietnam) 

David & Asamoah, 201 1 (Ghana) 

Price, 2001 (Philippines) 

Reddy & Suryamani, 2005 (India) 

Ali & Sharif, 201 1 (Pakistan) 

Mutandwa & Mpangwa, 2004 (Zimbabwe) 
Khan et at, 2007 (Pakistan) 

Erbaugh et at, 2010 (Uganda) 

Rebaudo & Dangles, 201 1 (Ecuador) 
Subtotal (l-squared = 92.1%, p = 0.000) 

Other FFS curriculum 
Waarts et at, 2012 (Kenya) 

David, 2007 (Cameroon) 

Endalew, 2009 (Ethiopia) 

Orozco Cirilo et at, 2008 (Mexico) 

Rao et at, 2012 (India) 

Kelemework, 2005 (Ethiopia) 

Dinpanah et at, 2010 (Iran) 

Bunyatta et at, 2006 (Kenya) 

Subtotal (l-squared = 94.6%, p = 0.000) 


0.14(0.03, 0.25) 
0.17(0.03,0.31) 
0.17(0.07, 0.27) 

1 0.42 (-0.17, 1.01) 

0.45 (-0.04, 0.94) 

0.46(0.18,0.73) 
— • — 0.59 (0.25, 0.92) 

■— 0.79(0.29,1.29) 

1.14(0.93,1.34) 

•— 1.79(1.17,2.41) 

O 0.56(0.31,0.80) 


(-0.03, 0.08) 
(0.17, 0.24) 
(-0.06, 0.60) 
(0.37, 0.47) 
(-0.02, 0.87) 
(-0.22, 1.29) 
(0.41, 0.92) 
(0.65, 1.41) 
(0.22, 0.54) 


0.03 

0.20 

0.27 

0.42 

0.43 

0.54 

0.67 

1.03 

0.38 


IPM neighbour farmers 

Khan et at, 2007 (Pakistan) ♦ 

Orozco Cirilo et at, 2008 (Mexico) ♦ 

Reddy & Suryamani, 2005 (India) I 

Ricker-Gilbert et at, 2008 (Bangladesh) — 

Rebaudo & Dangles, 201 1 (Ecuador) — 

Subtotal (l-squared = 5.2%, p = 0.377) C 


-0.13 (-0.68, 0.42) 
-0.07 (-0.07, -0.06) 
0.05 (-0.45, 0.56) 
0.17 (-0.25, 0.59) 
0.38 (-0.15, 0.91) 
-0.05 (-0.13, 0.03) 


0 .5 1 2 

Favours non- Favours FFS participants 


We attempted to explain the heterogeneity through sensitivity and moderators analysis. We 
examined whether the findings are sensitive to the types of studies included, risk of bias, 
effect size estimate and outcome measures, and length of follow-up (Table 6). 25 Results of the 
sensitivity analysis suggest that findings are not sensitive to choice of outcome measure. 
However, rigorous quasi- experimental studies (including double differences, propensity 
score matching and instrumental variables estimation) and multivariate regression studies 

24 The forest plots showing programme name rather than authors are in Appendix G. 

25 Exposure length is measured imprecisely in the original studies, so we have adopted cut-off values of up to one 
year, up to two years and greater than two years to distinguish longer- from shorter-term effects. 
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produce smaller effect sizes than other types of study designs using unadjusted analyses. 
Similarly, studies assessed as being of lower risk of bias also show smaller effects on average 
(SMD=0.21, 95% Cl =0.07, 0.35; Q^5, Tau-sq=0.008, 1-sq=55%; 3 observations) (Figure 14), 
as do SMD effect sizes which were calculated using Cox- transformed response ratios or odds 
ratios. 

Table 6 Sensitivity analysis of knowledge outcomes for FFS participants: study design 
characteristics 



SMD 

95% confidence interval 

Q 

Tau-sq 

l-sq 

P-value 

Sample 

size 

FFS participants: all studies 

0.46 

0.33 

0.58 

249 

0.050 

93.2% 

0.000 

18 

Study type: 

Rigorous quasi-experimental study 

0.21 

0.07 

0.35 

5 

0.008 

55.6% 

0.105 

3 

Cross-section regression (adjusted analysis) 

0.09 

-0.05 

0.23 

6 

0.009 

83.8% 

0.013 

2 

Longitudinal data (unadjusted analysis) 

0.43 

0.37 

0.48 

2 

0.000 

0.0% 

0.696 

5 

Cross-section data (unadjusted analysis) 

0.73 

0.37 

1.10 

134 

0.243 

94.8% 

0.000 

8 

Risk of bias: 

Medium risk of bias 

0.21 

0.07 

0.35 

5 

0.008 

55.6% 

0.105 

3 

High risk of bias 

0.52 

0.38 

0.67 

243 

0.054 

94.2% 

0.000 

15 

Outcome measure: 

Knowledge test score 

0.46 

0.33 

0.58 

246 

0.047 

93.9% 

0.000 

16 

Percentage with improved knowledge 
Num technologies known 

0.47 
No obs. 

0.21 

0.73 

0 

0.000 

0.0% 

0.850 

2 

Effect size calculation: 

SMD (standard calculation) 

0.73 

0.49 

0.97 

41 

0.112 

75.5% 

0.000 

11 

SMD (Cox adjusted OR/RR) 

0.46 

0.18 

0.73 

106 

0.022 

94.4% 

0.000 

7 

Length of follow-up data collection:* 

1 year or less 

0.19 

0.15 

0.24 

2 

0.000 

11.0% 

0.325 

3 

2 years or less 

0.43 

-0.08 

0.94 

19 

0.177 

89.5% 

0.000 

3 

More than 2 years 

0.46 

0.23 

0.69 

82 

0.070 

92.6% 

0.000 

7 


Note: * indicates data incomplete due to missing observations. 
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Figure 14 Knowledge outcomes for FFS participants by risk of bias status 


Study 

ID 

High risk of bias 
Waarts et at, 2012 (Kenya) 

David & Asamoah, 201 1 (Ghana) 

David, 2007 (Cameroon) 

Endalew, 2009 (Ethiopia) 

Orozco Cirilo et at, 2008 (Mexico) 

Price, 2001 (Philippines) 

Rao et at, 2012 (India) 

Reddy & Suryamani, 2005 (India) 
Kelemework, 2005 (Ethiopia) 

Mutandwa & Mpangwa, 2004 (Zimbabwe) 
Dinpanah et at, 2010 (Iran) 

Khan et at, 2007 (Pakistan) 

Bunyatta et at, 2006 (Kenya) 

Erbaugh et at, 2010 (U) 

Rebaudo & Dangles, 201 1 (Ecuador) 
Subtotal (l-squared = 94.2%, p = 0.000) 

Medium risk of bias 
Godtland et at, 2004 (Peru) 

Rejesus et at, 2010 (Vietnam) 

AN & Sharif, 2012 (Pakistan) 

Subtotal (l-squared = 55.6%, p = 0.105) 

Overall (l-squared = 93.2%, p = 0.000) 


T 


Favours non-FFS 



i 

X 

I 


I 

+ 

I 

I 

I 

I 

I 

I 


O 



o; 

i 


i 

i 

± 


ES (95% Cl) 

0.03 

(-0.03, 0.08) 

0.17 

(0.07, 0.27) 

0.20 

(0.17, 0.24) 

0.27 

(-0.06, 0.60) 

0.42 

(0.37, 0.47) 

0.42 

(-0.17, 1.01) 

0.43 

(-0.02, 0.87) 

0.45 

(-0.04, 0.94) 

0.54 

(-0.22, 1.29) 

0.59 

(0.25, 0.92) 

0.67 

(0.41,0.92) 

0.79 

(0.29, 1.29) 

1.03 

(0.65, 1.41) 

1.14 

(0.93, 1.34) 

- 1.79 

(1.17,2.41) 

0.52 

(0.38, 0.67) 


0.14 

(0.03, 0.25) 

0.17 

(0.03, 0.31) 

0.46 

(0.18, 0.73) 

0.21 

(0.07, 0.35) 

0.46 

(0.33, 0.58) 


0 

Favours FFS participants 


We also examined whether findings are moderated by characteristics of interventions and 
fanners (Table 7). Unfortunately, the information on FFS design and implementation was 
incomplete, both that reported in the impact evaluations themselves and other documents 
we were able to link with the impact evaluations such as those collected for the portfolio 
review (Appendix A) . We were therefore only able to obtain evidence suggesting FFS 
interventions were more effective in improving knowledge where they used "fanner practice" 
control plots or made attempts at institutionalisation in the local community such as through 
dedicated training of fanner trainers. Results do not suggest that FFS in which women 
fanners were known to participate are systematically associated with larger or smaller effects 
on knowledge; and the same applies with other characteristics of participants such as years 
of education (Table 7). We did, however, find more robust evidence that FFS interventions 
promoting I PM, as opposed to other curricula, tend to have bigger effects on participant 
knowledge, while FFS for permanent crops (including coffee, tea and cocoa) tend to have 
smaller effects on knowledge. The latter may be due to the method of implementation for 
permanent crop field schools, which are conducted only fortnightly rather than weekly, but 
we were not able to test for this possibility due to insufficient data. 
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Table 7 Moderator analysis of knowledge outcomes for FFS participants: intervention and farmer 
characteristics 




95% confidence interval 

Q 

Tau- 

l-sq 

P- 

Sample 






sq 


value 

size 

FFS participants: all studies 

0.46 

0.33 

0.58 

249 

0.050 

93.2% 

0.000 

18 

Curriculum: 









IPM 

0.63 

0.30 

0.96 

104 

0.192 

93.2% 

0.000 

8 

IPPM 

0.35 

-0.06 

0.75 

5 

0.070 

81.4% 

0.020 

2 

Other curricula 

0.38 

0.22 

0.54 

131 

0.035 

94.6% 

0.000 

8 

FFS +in put/marketing 

No obs. 









FFS design: 

Local curriculum development* 

Training-of-trainers programme for facilitators* 
Group formation* 

Participatory method of tuition* 

FAO involvement* 

Local institutionalisation incl training farmer trainers* 
No local institutionalisation/training farmer trainers* 


0.38 

0.25 

0.52 

207 

0.041 

94.7% 

0.000 

12 

0.24 

0.13 

0.34 

56 

0.015 

83.9% 

0.000 

10 

0.49 

0.27 

0.71 

105 

0.079 

93.4% 

0.000 

8 

0.42 

0.22 

0.62 

189 

0.074 

95.8% 

0.000 

9 

0.44 

0.21 

0.67 

11 

0.042 

64.7% 

0.023 

5 

0.35 

0.20 

0.49 

128 

0.030 

94.5% 

0.000 

8 

0.15 

0.05 

0.25 

1 

0.000 

0.0% 

0.469 

2 


FFS activities: 

Season-long weekly training* 

Uses "farmer practice" control plot* 

Does not use "farmer practice" control plot* 
Agro-ecosystems analysis* 

Group dynamics activities* 

Special topic* 

Exchange visits* 

Field day* 


0.24 

0.14 

0.33 

22 

0.009 

67.9% 

0.003 

8 

0.29 

0.17 

0.42 

26 

0.015 

77.2% 

0.000 

7 

0.03 

-0.03 

0.08 

0 

0.000 

n/a 

n/a 

1 

0.44 

0.27 

0.61 

107 

0.054 

91.6% 

0.000 

10 

0.22 

0.10 

0.34 

49 

0.015 

87.8% 

0.000 

7 

0.46 

0.34 

0.58 

132 

0.119 

95.5% 

0.000 

7 

0.47 

0.03 

0.91 

18 

0.133 

88.9% 

0.000 

3 

0.49 

0.26 

0.73 

105 

0.081 

94.3% 

0.000 

7 


Years of implementation of FFS programme: 


1 year or less 

0.31 

0.17 

0.44 

76 

0.024 

90.7% 

0.000 

8 

2 years or less 

0.51 

0.30 

0.73 

64 

0.074 

89.1% 

0.000 

8 

More than 2 years 

0.65 

0.37 

0.93 

0 

0.000 

0.0% 

0.502 

2 

Region: 









Central Asia (CA) 

0.67 

0.41 

0.92 

0 

0.000 

n/a 

n/a 

1 

East Asia (EA) 

0.19 

0.05 

0.32 

1 

0.000 

0.0% 

0.423 

2 

Latin America (LA) 

0.55 

0.21 

0.89 

41 

0.071 

95.1% 

0.000 

3 

South Asia (SA) 

0.50 

0.31 

0.70 

2 

0.000 

0.0% 

0.670 

4 

Sub-Saharan Africa (SSA) 

0.45 

0.26 

0.63 

142 

0.053 

95.1% 

0.000 

8 

Crop: 









Rice 

0.41 

0.11 

0.71 

11 

0.062 

73.7% 

0.010 

4 

0.78 

0.45 

1.10 

103 

0.130 

95.1% 

0.000 

6 

Other staples and vegetables 


Cotton 

0.54 

0.36 

0.72 

2 

0.000 

0.0% 

0.679 

4 

0.15 

0.03 

0.26 

27 

0.010 

89.0% 

0.000 

4 

Permanent (coffee, tea, cocoa) 



Farmer characteristics: 









Farmer education years exceeds national average* 

0.45 

0.22 

0.68 

87 

0.067 

94.3% 

0.000 

6 

Women farmers participated in FFS* 

0.44 

0.02 

0.85 

75 

0.168 

96.0% 

0.000 

4 


Note: * indicates data incomplete due to missing observations. 
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We conducted meta- regression analysis to assess robustness of variables which seem to 
explain differences in effect sizes in bivariate analysis (Table 8). Due to the limited sample 
size, we restricted the analysis to those variables which were associated with significant 
differences in effect sizes in bivariate analysis, and we included the log standard error 
(Egger's test coefficient) in specification 1, given possibilities for publication bias. 26 The 
analysis suggests variables significantly associated with smaller impacts on farmer 
knowledge included studies subject to medium risk of bias (as opposed to high risk of bias), 
measurements for neighbour farmers and FES which targeted permanent crops. The results 
were not sensitive to exclusion of insignificant variables (Table 8 specification 2, conducted 
given the low number of observations and consequently limited power of the analysis). The 
test statistics suggest that the model explains substantial heterogeneity between studies 
(adjusted R- squared and Tau- squared), although unexplained heterogeneity remains (I- 
squared). 


Table 8 Meta- regression analysis of knowledge outcomes 



Coefficient 

(i) 

t-statistic 

P>t 

Coefficient 

(2) 

t-statistic 

P>t 

l=Medium risk of bias 

■0.50 

■2.05 

0.057 

■0.49 

-3.03 

0.001 

1=C ox-transfo rmed SMD 

-0.26 

-1.13 

0.277 




1=IPM-FFS 

0.25 

1.49 

0.156 




l=Permanentcrop 

■0.38 

■2.57 

0.020 

■0.57 

-3.94 

0.001 

l=Neighbour fa rmers 

■0.67 

■4.37 

0.000 

-0.68 

-4.23 

0.000 

Standard error (Egger coefficient) 

-0.67 

-0.64 

0.530 




Constant 

1.12 

4.80 

0.000 

0.73 

8.11 

0.000 

Numberof obs 

23 



23 



Tau-squared 

0.019 



0.041 



l-squared 

72.1% 



82.7% 



Adjusted R-squared 

84.7% 



67.9% 



Model F(5, 17) 

6.91 



8.73 



P rob > F 

0.001 



0.000 




Note: Bold indicates coefficients statistically significant at <10% level. 


Few IPM-FFS studies included in our review assessed diffusion of IPM knowledge to non- 
participant neighbour farmers who were potentially exposed to the message; all of the 
studies which reported these outcomes were assessed as being of high risk of bias. The meta- 
analysis evidence does not suggest there are knowledge spillovers among the few studies 
which measured knowledge among neighbours (Figure 13) (SMD— 0.05, 95% Cl =-0.13, 
0.03; Q=4.2, Tau-sq=0.002, 1-sq=5%; 5 observations). The values of I-squared and Tau- 
squared indicate that this finding is highly consistent across studies, and do not suggest 


26 In order to maximise sample size, we also excluded variables with insufficient observations such as FFS which 
incorporated local institutionalisation or training of farmer trainers. 
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there are contexts in which spillovers to neighbours are more likely. Moreover, as noted 
above, we also found evidence suggestive of file-drawer effects in terms of under-reporting of 
insignificant or negative findings. For example, despite reporting increases in knowledge 
among FFS participants over comparison farmers (RR=1. 14), Rola et al. (2002) reported no 
difference in knowledge between exposed and comparison farmers (RR=0.99); however, 
they did not report on statistical precision of their findings. In a similar study which 
demonstrated knowledge improvements among FFS participants but did not present 
information to estimate statistical precision of findings, Tripp et al. (2005) reported limited 
knowledge among exposed farmers based on both insecticide knowledge test scores 
(RR=0.90) and number of natural enemies mentioned (RR=0.94). Feder et al. (2004) could 
not detect a statistically significant effect on knowledge diffusion to FFS- exposed farmers in 
Indonesia, and consequently did not report findings. 

Some studies examined different types of knowledge diffusion. In the single study in 
Bangladesh which examined diffusion of "simple" and more "complex" practices, Ricker- 
Gilbert et al. (2008) found that non- participant neighbour farmers living in FFS villages 
were significantly more likely to have knowledge of simple I PM practices - such as placing 
branches in rice fields for birds to perch o - than comparison farmers living in non- FFS 
villages who were exposed to other sources of information on IPM (Appendix G). However, 
neighbours were not more likely than comparison farmers to have knowledge of 
intermediate or more complex IPM practices such as setting insect traps and using beneficial 
insects, which FFS farmers were shown to have, suggesting spillovers to neighbours may be 
possible for simple practices only, over and above IPM marketing information. 27 

As several studies noted, not all the knowledge in the FFS curriculum is observable or can be 
learned through self-study by non- participants and it is difficult to transmit complex 
knowledge such as agro- ecosystem concepts, analysis and decision-making principles 
through conversation alone (Feder et al., 2004; Pananurak, 2010; Wu, 2010). Due to the 
limited heterogeneity in findings for FFS neighbours, and limited reporting of sub-groups in 
primaiy studies, it is not possible to test whether there are systematic differences in 
spillovers across studies based on farmer characteristics. 28 


Adoption of improved practices (intermediate outcomes) 


27 Rejesusetal. (2010) found that there was greater acquisition for some aspects of knowledge (such as 
entomological knowledge) among fanners with lower initial levels of knowledge about IPM. The same study finds 
that the initial gains in knowledge observed in FFS graduates compared with exposed farmers do not persist over 
time. However, it is not possible to infer whether this was due to diffusion within communities, or erosion of the 
FFS graduate's knowledge over time, or indeed due to some other reasons such as changes in the implementation 
of the proj ect over time, resulting in differing levels of effectiveness across the years that were included in the 
analysis. 

28 One study in Indonesia used statistical network regression analysis to find diffusion was maximised at an 
optimal level of social 'superiority' of opinion leaders, when opinion leaders who received intensive training are 
slightly superior to 'would-be followers' in terms of socioeconomic status and farming skill attributes, but not 
excessively so. In contrast, when the social or educational 'distance' between opinion leaders and followers was 
too large, their effectiveness in diffusing skills and knowledge fell (Feder & Savastano, 2006). 
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Adoption of practices is the most frequent outcome measure reported in the studies. We 
report two main types of adoption variables: variables measuring pesticide use for FFS 
programmes involving IPM and IPPM, and variables measuring adoption of other improved 
practices and labour costs (detail on outcomes variables is reported in Appendix E) . 

Pesticide use for IPM/ IPPM farmer field schools 

Measures of pesticide use usually took the form of the volume of pesticides used per unit of 
land area or the number of sprays per cropping season, or total pesticide expenditure. For 
these outcomes, we have estimated response ratio (RR) effect sizes. Reduced pesticide 
demand is considered a beneficial outcome, since all interventions included in this meta- 
analysis promoted IPM or IPPM farmer field schools (with or without complementary input 
and/ or marketing interventions) . Reductions in pesticide use - and therefore positive 
impacts of FFS training - are measured as values of RR between 0 and 1. Increases in 
pesticide use over the comparison group are measured as values of RR greater than 1 RR are 
interpretable as percentage changes over the comparison. 

The meta- analysis suggests that demand for pesticides, measured in terms of amount 
sprayed or total expenditure, is estimated on average to be significantly lower among 
IPM/ IPPM FFS graduates than comparison farmers (RR=0.68, 95% Cl =0.57, 0.81; Q=172, 
Tau-sq=0.09, 1-sq=90%; 18 observations), but not IPM/ IPPM FFS interventions which 
included additional intervention components of input or marketing support (RR=0.77, 95% 
Cl =0.30, 1.96; Q=158, Tau-sq=0.90, 1-sq=98%; 4 observations) (Figure 15). Furthermore, 
and commensurate with the lack of knowledge gains reported above, we did not find 
evidence for diffusion to neighbouring farmers, estimating non- significant reductions in 
pesticide demand for this group on average (RR=0.91, 95% C3 =0.66, 1.26; Q=47, Tau- 
sq=0.12, 1-sq=85.2%; 8 observations). 29 

There is, however, substantial variation in effect sizes which appears to be in large part due 
to heterogeneity across studies (indicated statistically by high values of I-squared and Tau- 
squared). We therefore also conducted additional sensitivity and moderator analyses to 
explore the heterogeneity. Similarly to knowledge outcomes, we found that less rigorous 
study designs (Table 9) and high- risk- of- bias studies were likely to overestimate impacts on 
pesticide adoption (Figure 16 reports the forest plot excluding high- risk- of- bias studies). 
Indeed the best estimate of average effects on pesticide use for IPM/ IPPM curricula FFS 
suggests a reduction in pesticide use of 23 per cent on average (RR=0. 77, 95% Cl =0.61, 0.97; 
Q=40, Tau-sq=0.07, 1-sq=83%; observations=8). Impacts measured over shorter-term 
follow-up periods (up to two years) tended to be bigger than longer-term periods, as did 
studies which measured pesticide costs. The heterogeneity analysis for pesticide use by 
neighbouring farmers suggests results are not sensitive to dropping high- risk-of- bias studies 
(RR=0.95, 95% Cl =0.64, 1.39; Q=45, Tau-sq=0.14, 1-sq=91%, 5 observations). Results are 
also not sensitive to exclusion of the studies by Feder et al. (2004) and Yamazaki and 

29 The forest plot showing programme name rather than authors is in Appendix G. 
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Resosudarmo (2008) (see also forest plots in Appendix G). 


Figure 15 Pesticide use for I PM/ IPPM FFS participants and IPM neighbours versus non-FFS 
comparison farmers 


Study 

ID 


I PM/I PPM FFS 


Yamazaki & Resosudarmo, 2008 (In dones i a) — 
Yorobe et at, 201 1 (Philippines) 

Yang et at, 2005 (China) 

Khalid, n.d. (Sudan) 

Rejesus et al., 2010 (Vietnam) 

Pananurak, 2010 (India) 

Mutandwa & Mpangwa, 2004 (Zimbabwe) 
Pananurak, 2010 (Pakistan) 

Amera, 2009 (Ethiopia) 

Mancini et at, 2008 (India) 

Wu, 2010 (China) 

Van den Berg & Amarasinghe, 2002 (Sri Lanka) 
Praneetvatakul & Waibel, 2006 (Thailand) 
Murphy et at, 2002 Vietnam) 

AN & Sharif, 201 1 (Pakistan) 

Khan et at, 2007 (Pakistan) 

Labarta, 2005 (Nicaragua) 

Feder et at, 2004 (Indonesia) 

Subtotal (l-squared = 90.1%, p = 0.000) 

FFS+input/marketing 
Birthal et at, 2000 (India) 

Cole et at, 2007 (Ecuador) 

Cavatassi et at, 201 1 (Ecuador) 

Friis-Hansen et at, 2004 (Uganda) 

Subtotal (l-squared = 98.1%, p = 0.000) 

IPM neighbour farmers 
Pananurak, 2010 (India) 

Yamazaki & Resosudarmo, 2007 (Indonesia) 
Wu, 2010 (China) 

Pananurak, 2010 (Pakistan) 

Labarta, 2005 (Nicaragua) 

Praneetvatakul & Waibel, 2006 (Thailand) 

Khan et at, 2007 (Pakistan) 

Feder et at, 2004 (Indonesia) 

Subtotal (l-squared = 85.2%, p = 0.000) 



.1 .25 .5 1 2 3 


Favours FFS participants Favours non-FFS 


RR (95% Cl) 

0.20 1 

[0.01, 

3.23) 

0.37 1 

[0.18, 

0.78) 

0.41 | 

[0.36, 

0.46) 

0.48 1 

[0.31, 

0.75) 

0.52 | 

[0.24, 

1.12) 

0.52 | 

[0.30, 

0.92) 

0.57 1 

[0.36, 

0.89) 

0.59 1 

[0.41, 

0.87) 

0.61 1 

[0.52, 

0.71) 

0.67 1 

[0.46, 

0.97) 

0.71 | 

[0.64, 

0.80) 

0.82 | 

[0.74, 

0.90) 

0.82 | 

[0.68, 

0.98) 

0.83 1 

[0.75, 

0.93) 

0.90 1 

[0.75, 

1.09) 

0.91 1 

[0.28, 

2.94) 

0.95 1 

[0.39, 

2.34) 

1.30 1 

[1.08, 

1.57) 

0.68 1 

[0.57, 

0.81) 

0.21 (0.17, 

0.26) 

0.88 1 

[0.68, 

1.13) 

1.34 1 

[0.99, 

1.80) 

1.42 (1.09, 

1.86) 

0.77 (0.30, 

1.96) 

0.54 1 

[0.25, 

1.15) 

0.67 1 

[0.12, 

3.88) 

0.68 1 

[0.62, 

0.76) 

0.78 1 

[0.40, 

1.49) 

0.99 1 

[0.42, 

2.33) 

1.151 

[0.92, 

1.43) 

1.20 1 

[0.40, 

3.53) 

1.30 1 

[1.09, 

1.55) 

0.91 1 

[0.66, 

1.26) 


Table 9 Sensitivity analysis: pesticide use for IPM / IPPM FFS participants and neighbours 




95% confidence interval 

Q 

Tau-sq 

l-sq 

P-value 

Sample 

size 

1. FFS participants: all studies 

0.691 

0.571 

0.836 

331 

0.161 

93.7% 

0.000 

22 

Study type: 









Rigorous quasi-experimental study 

0.804 

0.650 

0.994 

53 

0.077 

81.1% 

0.000 

11 

Cross-section regression (adjusted analysis) 

No obs. 








Longitudinal data (unadjusted analysis) 

0.687 

0.482 

0.979 

0 

0.000 

0.0% 

0.630 

2 

Cross-section data (unadjusted analysis) 

0.614 

0.45 

0.839 

243 

0.210 

96.7% 

0.000 

9 
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RR 

95% confidence interval 

Q 

Tau-sq 

l-sq 

P-value 

Sample 

size 

Risk of bias: 









Medium risk of bias 

0.825 

0.658 

1.036 

245 

0.200 

95.1% 

0.000 

13 

Medium risk of bias (IP M/IP P M only) 

0.772 

0.613 

0.974 

40 

0.069 

82.7% 

0.000 

8 

Medium risk of bias (excl. Feder, Yamazaki studies) 

0.783 

0.640 

0.957 

245 

0.200 

95.1% 

0.000 

11 

High risk of bias 

0.632 

0.484 

0.826 

50 

0.078 

84.0% 

0.000 

9 

Outcome measure: 









Pesticide use* 

0.726 

0.581 

0.907 

106 

0.121 

89.6% 

0.000 

12 

Pesticide costs* 

0.584 

0.433 

0.788 

248 

0.218 

95.6% 

0.000 

12 

Length of follow-up:* 









2 years or less 

0.633 

0.475 

0.842 

211 

0.209 

94.8% 

0.000 

12 

More than 2 years 

0.780 

0.610 

0.998 

67 

0.095 

88.0% 

0.000 

9 


2. FFS neighbours: all studies 

0.913 

0.663 

1.257 

47 

0.123 

85.2% 

0.000 

8 

Study type: 









Rigorous quasi-experimental study 

0.896 

0.641 

1.251 

47 

0.127 

87.2% 

0.000 

7 

Cross-section regression (adjusted analysis) 

No obs. 








Longitudinal data (unadjusted analysis) 

1.196 

0.405 

3.534 

0 

0.000 

n/a 

n/a 

1 

Cross-section data (unadjusted analysis) 

No obs. 








Risk of bias: 









Medium risk of bias 

0.945 

0.644 

1.388 

45 

0.136 

91.2% 

0.000 

5 

Medium risk of bias (excl. Feder, Yamazaki studies) 

0.854 

0.564 

1.294 

17 

0.106 

88.4% 

0.000 

3 

High risk of bias 

0.789 

0.478 

1.303 

2 

0.000 

0.0% 

0.407 

3 

Outcome measure: 









Pesticide use* 

0.645 

0.488 

0.852 

1 

0.014 

11.9% 

0.287 

2 

Pesticide use (high risk of bias excluded) 

No obs. 








Pesticide costs 

0.905 

0.644 

1.272 

47 

0.129 

87.3% 

0.000 

7 

Pesticide costs (high risk of bias excluded) 

0.945 

0.644 

1.388 

45 

0.136 

91.2% 

0.000 

5 

Length of follow-up:* 









2 years or less 

0.691 

0.624 

0.765 

2 

0.000 

0.0% 

0.696 

5 

2 years or less (high risk of bias excluded) 

0.687 

0.619 

0.762 

0 

0.000 

0.0% 

0.710 

2 

More than 2 years 

1.231 

1.073 

1.413 

1 

0.000 

0.0% 

0.550 

3 

More than 2 years (high risk of bias excluded) 

1.231 

1.073 

1.413 

1 

0.000 

0.0% 

0.550 

3 

Note: *2 studies measuring both pesticide use and costs (Khan etal, 2007; Yang etal., 2005) are included in sub-group analysis by outcome measure. 


We conducted additional analyses of moderators to explore the variation in findings across 
studies according to intervention and participating fanner characteristics (Table 10). The 
results suggest that the intervention characteristics which are associated with systematically 
smaller and insignificant impacts on pesticide use programmes implemented at national 
scale, programmes which involved complementary input or marketing components (as 
opposed to "pure" I PM/ IPPM), and programmes in Latin America; in contrast, the results 
suggest impacts on pesticide use were biggest for cotton crops. We did not find differences 
for other variables describing intervention activities or length of implementation, with the 
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exception of lack of impact for the single FFS for which it was clear that "fanner practice" 
control plots were not implemented (Labarta, 2005) . However, the analysis by 
characteristics of participating farmers suggests socioeconomic status may affect pesticide 
adoption, as measured by education levels and landholdings. Field schools targeting women 
also seemed to have lower levels of adoption, although due to limited data available on many 
of the interventions and farmer characteristics variables, these results should be interpreted 
cautiously. 

Figure 16 Pesticide use for FFS participants and neighbours excluding high- risk-of- bias studies 


Study 



Response ratio 

ID 



ES (95% Cl) 

IPM/IPPM FFS 




Yamazaki & Resosudarmo, 2008 (Indonesia) 



0.20 (0.01,3.23) 

Yorobe et al., 201 1 (Philippines) 



0.37 (0.18, 0.78) 

Rejesus et al., 2010 (Vietnam) 



0.52 (0.24, 1.12) 

Pananurak, 2010 (Pakistan) 

-H- 


0.59 (0.41, 0.87) 

Wu, 2010 (China) ' 

♦ 


0.71 (0.64, 0.80) 

Praneetvatakul & Waibel, 2006 (Thailand) 



0.82 (0.68, 0.98) 

Ali & Sharif, 201 1 (Pakistan) 


0.90 (0.75, 1.09) 

Feder et al., 2004 (Indonesia) 



1.30 (1.08, 1.57) 

Subtotal (l-squared = 82.7%, p = 0.000) 

o 

1 

0.77 (0.61, 0.97) 

FFS+input/marketing 




Cavatassi et al., 201 1 (Ecuador) 



1.34 (0.99, 1.80) 

Subtotal (l-squared = .%, p = .) 


o 

1.34 (0.99, 1.80) 

IPM neighbour farmers 




Yamazaki & Resosudarmo, 2008 (Indonesia) 



0.67 (0.12, 3.88) 

Wu, 2010 (China) 

♦ 


0.68 (0.62, 0.76) 

Pananurak, 2010 (Pakistan) 



0.78 (0.40, 1.49) 

Praneetvatakul & Waibel, 2006 (Thailand) 



1.15 (0.92, 1.43) 

Feder et al., 2004 (Indonesia) 



1.30 (1.09, 1.55) 

Subtotal (l-squared = 91 .2%, p = 0.000) 

< 

> 

0.95 (0.64, 1.39) 

1 1 1 
.1 .25 .5 

ru 

1 2 3 


Favours FFS participants Favours non-FFS 


Table 10 Moderator analysis of pesticide use for IPM J IPPM FFS participants: intervention and 
farmer characteristics 




95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P- 

value 

Sample 

size 

FFS participants: all studies 

0.691 

0.571 

0.836 

331 

0.161 

93.7% 

0.000 

22 

Curriculum: 









IPM-FFS 

0.688 

0.576 

0.822 

171 

0.094 

91% 

0.000 

17 

IPPM-FFS 

0.567 

0.362 

0.888 

0 

0.000 

n/a 

n/a 

1 

IPM/IPPM FFS+complementary inputs/marketing 

0.767 

0.300 

1.958 

158 

0.897 

98% 

0.000 

4 


FFS design: 
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RR 

95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P- 

value 

Sample 

size 

Local curriculum development* 

0.687 

0.551 

0.858 

317 

0.176 

95% 

0.000 

18 

Training-of-trainers programme for facilitators* 

0.690 

0.555 

0.859 

317 

0.175 

95% 

0.000 

18 

Group formation* 

0.725 

0.587 

0.894 

175 

0.116 

93% 

0.000 

14 

Participatory method of tuition* 

0.838 

0.626 

1.122 

43 

0.108 

84% 

0.000 

8 

Local institutionalisation including training of farmer trainers* 

0.695 

0.549 

0.882 

302 

0.168 

96% 

0.000 

14 

No local institutionalisation or training of farmer trainers* 

0.576 

0.230 

1.441 

2 

0.264 

60% 

0.114 

2 

FAO involvement* 

0.761 

0.629 

0.920 

204 

0.118 

92% 

0.000 

17 

FFS activities: 

Season-long weekly training* 

0.631 

0.496 

0.803 

134 

0.137 

91% 

0.000 

13 

Uses "farmer practice" control plot* 

0.729 

0.553 

0.962 

163 

0.168 

94% 

0.000 

11 

Does not use "farmer practice" control plot* 

0.954 

0.389 

2.339 

0 

0.000 

n/a 

n/a 

1 

Agro-ecosystem analysis* 

0.726 

0.596 

0.883 

51 

0.064 

78% 

0.000 

12 

Group dynamics activities* 

0.754 

0.609 

0.934 

45 

0.064 

85% 

0.000 

8 

Special topic* 

0.732 

0.581 

0.922 

177 

0.136 

93% 

0.000 

13 

Exchange visits* 

0.815 

0.758 

0.876 

3 

0.000 

0% 

0.394 

4 

Field day* 

0.767 

0.619 

0.952 

184 

0.113 

94% 

0.000 

13 

Years of implementation of FFS programme: 

2 years or less 

0.672 

0.527 

0.856 

283 

0.188 

95% 

0.000 

14 

More than 2 years 

0.691 

0.571 

0.836 

39 

0.121 

82% 

0.000 

8 

Scale of implementation of FFS programme: 

Programme implemented at national scale 

0.779 

0.322 

1.887 

7 

0.374 

71% 

0.033 

3 

Pilot project or regional programme 

0.675 

0.556 

0.819 

286 

0.149 

94% 

0.000 

19 

Region: 

Central Asia (CA) 

No obs. 








East Asia (EA) 

0.680 

0.495 

0.934 

147 

0.155 

95% 

0.000 

8 

Latin America (LA) 

1.057 

0.761 

1.468 

5 

0.044 

56% 

0.105 

3 

South Asia (SA) 

0.587 

0.371 

0.927 

128 

0.327 

95% 

0.000 

7 

Sub-Saharan Africa (SSA) 

0.709 

0.432 

1.163 

34 

0.225 

91% 

0.000 

4 

Crop: 

Rice 

0.887 

0.677 

1.161 

23 

0.056 

82% 

0.000 

5 

Other staples and vegetables 

0.876 

0.667 

1.151 

33 

0.093 

82% 

0.000 

7 

Cotton 

Permanent crops 

0.550 
No obs. 

0.415 

0.729 

148 

0.169 

94% 

0.000 

10 

FFS farmer characteristics: 

Women farmers participated in FFS* 
Women farmers did not participate in FFS* 

0.880 
No obs. 

0.618 

1.254 

38 

0.122 

87% 

0.000 

6 

Farmer education years (greater than national average)* 
Farmer education years (not greater than national average)* 

0.647 
No obs. 

0.454 

0.922 

122 

0.208 

93% 

0.000 

9 

Farmer education years (greater than comparison)* 

0.699 

0.630 

0.777 

2 

0.000 

0% 

0.659 

4 

Farmer education years (not greater than comparison)* 

0.723 

0.391 

1.335 

140 

0.480 

96% 

0.000 

6 

Landholdings larger than comparison* 

0.613 

0.366 

1.027 

35 

0.299 

86% 

0.000 

6 

Landholdings smaller than comparison* 

0.814 

0.442 

1.499 

17 

0.282 

82% 

0.001 

4 


Note: * indicates data incomplete due to missing observations. 
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Sensitivity analysis for IPM-FFS neighbour farmers suggests that there may be diffusion in 
terms of reduced pesticide use for communities growing cotton over the short term (follow- 
up periods less than two years after FFS training), where FFS farmers are relatively highly 
educated. This finding is robust to exclusion of high- risk-of- bias studies (Table 11). The 
overall results also suggest diffusion effects among FFS communities in which participants 
had larger landholdings, although these findings are neither statistically significant nor 
robust to exclusion of high- risk-of- bias studies. It does not seem unreasonable to expect 
there to be diffusion of simple practices like reduced pesticide use due to interaction between 
FFS graduates and their communities, at least in the short term; indeed, as noted above, one 
study did find evidence for diffusion of "simple" knowledge (Ricker-Gilbert et al., 2008). 
However, those studies which examined sustainability found that any initial adoption among 
neighbouring farmers in terms of pesticide use fell within a few years (e.g.Wu, 2010, in 
China; Pananurak, 20 10, in India) . In addition, evidence from the one programme 
implemented at scale does not suggest any effects on diffusion (the Indonesia National IPM 
Programme evaluated by Feder et al., 2004 and Yamazaki & Resosudarmo, 2008) . The 
findings in terms of diffusion should be interpreted with caution due to the limited sample 
size and problems of confounding in bivariate analysis. We also examined heterogeneity for 
FFS neighbour groups using intervention design variables which might reasonably be 
expected to foster diffusion (i.e. farmer exchanges and field days, efforts at local 
institutionalisation, FAO involvement, years of implementation of the programme), but did 
not find systematic differences across studies according to these characteristics, since many 
were poorly reported in the included studies and project documents we not available. 


Table 11 Moderator analysis of pesticide use for IPM- FFS neighbours: intervention and farmer 
characteristics 




95% confidence 
interval 

Q 

Tau- 

sq 

l-sq 

P- 

value 

Sample 

size 

IPM-FFS neighbours: all studies 

0.913 

0.663 

1.257 

47 

0.123 

85% 

0.000 

8 

FFS design and activities: 

Local institutionalisation incl training of farmer trainers* 

0.924 

0.636 

1.342 

47 

0.137 

89% 

0.000 

6 

No local institutionalisation or training of farmer trainers* 

0.989 

0.421 

2.327 

0 

0.000 

n/a 

n/a 

1 

FAO involvement* 

0.905 

0.644 

1.272 

47 

0.129 

87% 

0.000 

7 

Exchange visits* 

0.848 

0.505 

1.425 

0 

0.000 

0% 

0.658 

2 

Field day* 

0.966 

0.688 

1.357 

46 

0.126 

87% 

0.000 

7 

Years of implementation of FFS programme: 

2 years or less 

1.147 

0.920 

1.429 

0 

0.000 

n/a 

n/a 

1 

More than 2 years 

0.861 

0.588 

1.261 

39 

0.149 

85% 

0.000 

7 

Scale of implementation of FFS programme: 

P rogramme implemented at national scale 

1.288 

1.080 

1.536 

1 

0.000 

0% 

0.462 

2 

P ilot project or regional programme 

0.841 

0.612 

1.157 

19 

0.083 

74% 

0.002 

6 

Region: 

Central Asia (CA) 
East Asia (EA) 

No 

obs. 

0.980 

0.637 

1.507 

45 

0.145 

93% 

0.000 

4 
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RR 

95% confidence 

Q 

Tau- 

l-sq 

P- 

Sample 



interval 


sq 


value 

size 

Latin America (LA) 

0.989 

0.421 

2.327 

0 

0.000 

n/a 

n/a 

1 

South Asia (SA) 

0.736 

0.470 

1.152 

1 

0.000 

0% 

0.488 

3 


No 








Sub-Saharan Africa (SSA) 

obs. 








Crop: 









Rice 

1.231 

1.073 

1.413 

1 

0 

0% 

0.550 

3 

Other staples and vegetables 

0.989 

0.421 

2.327 

0 

0 

n/a 

n/a 

1 

Cotton 

0.687 

0.620 

0.761 

2 

0 

0% 

0.675 

4 


No 








Permanent crops 

obs. 








FFS farmer characteristics: 









Women farmers participated in FFS* 

1.288 

1.080 

1.536 

1 

0.000 

0% 

0.462 

2 


No 








Women farmers did not participate* 

obs. 








F F S farmer education years (greater than national average)* 

0.879 

0.537 

1.440 

4 

0.173 

92% 

0.000 

4 

FFS farmer education years (greater than comparison)* 

0.690 

0.622 

0.765 

1 

0.566 

0% 

0.000 

3 

FFS farmer education years (not greater than comparison)* 

FFS farmer education years (greater than comparison) (excluding 

0.891 

0.446 

1.781 

5 

0.222 

63% 

0.069 

3 

high-risk-of-bias studies)* 

0.687 

0.619 

0.762 

0.1 

0.000 

0% 

0.710 

2 

FFS farmer education years (not greater than comparison) 
(excluding high-risk-of-bias studies)* 

1.288 

1.080 

1.536 

0.1 

0.000 

0% 

0.460 

2 

Landholdings larger than comparison* 

0.613 

0.366 

1.027 

35 

0.299 

86% 

0.000 

6 

Landholdings smaller than comparison* 

0.814 

0.442 

1.499 

17 

0.282 

82% 

0.001 

4 

Landholdings larger than comparison (excl high risk of bias)* 

0.625 

0.200 

1.960 

12 

0.704 

83% 

0.003 

3 

Landholdings smaller than comparison (excl high risk of bias)* 

0.578 

0.411 

0.812 

0.1 

0.000 

0% 

0.751 

2 


Note: * indicates data incomplete due to missing observations. 


Finally, we explored robustness of effect size moderators in multivariate meta- regression 
analysis (Table 12). The analysis across the full sample (specification 1) suggests studies 
which were assessed as being of high risk of bias, which targeted cotton crops, were 
significantly associated with larger effects on pesticide use, while neighbour farmers had 
significantly smaller effects. We did not find significant correlations between effect size and 
whether studies measured outcomes by pesticide costs (rather than use), length of 
intervention and follow-up, or programme scale (results not reported). In a second meta- 
regression analysis for the subsample of studies in which education data were available 
(specification 2), we also tested for the association between years of education (continuous 
variable) and variation in effects on pesticide use. Due to the limited sample size we included 
only those variables found to be statistically significant in specification 1. The results suggest 
that FFS which targeted fanners with a relatively greater number of years of education, and 
those targeting cotton crops, were significantly associated with bigger decreases in pesticide 
use, both for FFS participants and IPM-FFS neighbours. 

Table 12 Meta- regression analysis of pesticide use outcomes 
^ ( 1 ) ( 2 ) 
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Coefficient* 

t-statistic 

P>t 

Coefficient* 

t-statistic 

P>t 

l=Medium risk of bias 

0.315 

2.21 

0.035 

0.348 

3.57 

0.003 

l=Cotton crop 

■ 0.463 

■ 3.64 

0.001 

■ 0.364 

■ 2.44 

0.028 

l=Neighbour fa rmers 

0.238 

1.56 

0.130 




l=Pesticide costs 

-0.205 

-1.48 

0.149 




Average years of education 




■ 0.147 

■ 2.34 

0.033 

Constant 

-0.147 

-1.27 

0.214 

0.710 

1.87 

0.081 

Numberof obs 


33 



19 


Tau-squared 


0.076 



0.007 


l-squared 


75.4% 



37.0% 


Adjusted R-squared 


58.3% 



95.1% 


Model F 


6.62 



28.28 


P rob > F 


0.001 



0.000 



Notes: * coefficient estimates reported as natural logarithm. Bold indicates coefficient statistically significant at <10% level. 


Other reported measures of practices adopted 

Many studies, including those which were assessing field schools providing training in 
technologies other than IPM/ IPPM, reported measures of numbers of improved practices 
adopted. Studies used a range of outcomes including indices of adoption and numbers of 
practices adopted, and probability of farmers adopting positive practices (Appendix E). 
Similar to the variables used to measure knowledge outcomes, given the lack of a natural 
scale in many such outcomes, we considered it appropriate to calculate standardised mean 
difference effect sizes. 

The forest plot shows similar positive average impacts on adoption among FFS participants 
(SMD=0.63, 95% Cl =0.32, 0.94; Q=3,192, Tau-sq=0.314, 1-sq=99.6%; 14 observations) as 
compared with non- FFS farmers across the whole sample (Figure 17) . 30 The findings do not 
suggest differences on average for IPM and other (non-IPM/ IPPM) FFS curricula. No studies 
estimated effects on I PM- FFS neighbour farmers. The very high level of variability in effect 
sizes is likely to substantial between- study heterogeneity. Similar to other outcomes, we 
found medium risk of bias studies tended to produce effects of smaller magnitude 
(SMD=0.22, 95% Cl =0.06, 0.38; Q=10, Tau-sq=0.02, 1-sq=80%, 3 observations) than high 
risk of bias studies (Figure 18). Other factors relating to study design, effect size calculation 
and implementation appeared to be associated with variation in effects (Table 13), but we 
were unable to explain the variation in effects further using meta- regression analysis (results 
not reported). 


30 The forest plot showing programme names rather than authors is in Appendix G. 
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Figure 17 Beneficial practices adopted by IPM and other curricula FFS- participants 


Study Standardised mean difference 

ID ES (95% Cl) 


IPM/IPPM FFS 


Norvell & Hammig, 1999 (Indonesia) 

Ali & Sharif, 201 1 (Pakistan) 
Flicker-Gilbert et at, 2008 (Bangladesh) 
Amera, 2008 (Ethiopia) 

Mauceri et at, 2007 (Ecuador) 

Subtotal (l-squared = 99.9%, p = 0.000) 



0.08 (0.07, 0.09) 
0.22 (0.17, 0.27) 
0.92 (0.29, 1.55) 
1.19(1.15, 1.23) 
1.41 (0.96, 1.86) 
0.74 (0.18, 1.30) 


Other FFS 

Todo & Takahashi, 201 1 (Ethiopia) 
Friis-Hansen et at, 2004 (Uganda) 

Rao et at, 2012 (India) 

Endalew, 2009 (Ethiopia) 

Van Rijn, 2010 (Peru) 

Orozco Cirilo et at, 2008 (Mexico) 
Kelemework, 2005 (Ethiopia) 

Bunyatta et at, 2006 (Kenya) 

Dinpanah et at, 2010 (Iran) 

Subtotal (l-squared = 94.6%, p = 0.000) 


o 


0.06 (-0.06, 0.19) 
0.10 (-0.14, 0.33) 
0.11 (-0.33, 0.55) 
0.24 (-0.09, 0.58) 
0.46 (0.23, 0.69) 
0.64 (0.58, 0.70) 
0.70 (-0.28, 1 .67) 
1.45(1.04, 1.85) 
1.46(1.18, 1.73) 
0.56 (0.27, 0.86) 


Overall (l-squared = 99.6%, p = 0.000) 


0.63 (0.32, 0.94) 


0 .5 1 2 

Favours non-FFS Favours FFS participants 
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Figure 18 Beneficial practices adopted by risk of bias status 


Study Standardised mean difference 

ID ES (95% Cl) 


High risk of bias 

Norvell & Hammig, 1999 (Indonesia) 
Friis-Hansen et al., 2004 (Uganda) 

Rao et at, 2012 (India) 

Endalew, 2009 (Ethiopia) 

Orozco Cirilo et al., 2008 (Mexico) 
Kelemework, 2005 (Ethiopia) 
Ricker-Gilbert et at, 2008 (Bangladesh) 
Amera, 2008 (Ethiopia) 

Mauceri et al., 2007 (Ecuador) 

Bunyatta et at, 2006 (Kenya) 

Dinpanah et at, 2010 (Iran) 

Subtotal (l-squared = 99.7%, p = 0.000) 



4 - 



0.08 (0.07, 0.09) 
0.10 (-0.14, 0.33) 
0.11 (-0.33, 0.55) 
0.24 (-0.09, 0.58) 
0.64 (0.58, 0.70) 
0.70 (-0.28, 1 .67) 
0.92 (0.29, 1.55) 
1.19(1.15, 1.23) 
1.41 (0.96, 1.86) 
1.45(1.04, 1.85) 
1.46(1.18, 1.73) 
0.75 (0.33, 1.17) 


Medium risk of bias 

Todo & Takahashi, 201 1 (Ethiopia) 

Ali & Sharif, 201 1 (Pakistan) 

Van Rijn, 2010 (Peru) 

Subtotal (l-squared = 79.7%, p = 0.007) 


o 


0.06 (-0.06, 0.19) 
0.22 (0.17, 0.27) 
0.46 (0.23, 0.69) 
0.22 (0.06, 0.38) 


Overall (l-squared = 99.6%, p = 0.000) 


0.63 (0.32, 0.94) 


T 


T 


0 .5 1 2 

Favours non-FFS Favours FFS participants 


Table 13 Sensitivity analysis: other adoption measures 




95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P-value 

Sample 

size 

FFS participants: all studies 
FFS neighbours: all studies 

0.630 
No obs. 

0.321 

0.938 

3192 

0.314 

99.6% 

0.000 

14 

Study type: 

Longitudinal data, DID analysis 

0.064 

-0.064 

0.191 

0 

0.000 

n/a 

n/a 

1 

Longitudinal data (unadjusted analysis) 

0.641 

0.583 

0.699 

0.01 

0.000 

0% 

0.913 

2 

Cross-section data, adjusted analysis 

0.390 

0.215 

0.565 

82 

0.025 

95.1% 

0.000 

5 

Cross-section data (unadjusted analysis) 

0.764 

0.274 

1.254 

138 

0.350 

96.4% 

0.000 

6 

Risk of bias: 

Medium risk of bias 

0.223 

0.064 

0.381 

10 

0.015 

79.7% 

0.007 

3 

High risk of bias 

0.748 

0.330 

1.166 

3180 

0.458 

99.7% 

0.000 

11 

Outcome measure: 

Index of adoption of practices 

0.654 

0.280 

1.029 

489 

0.197 

99.0% 

0.000 

6 

Probability of adopting positive practices 

0.575 

0.072 

1.078 

1047 

0.425 

99.4% 

0.000 

7 

Num positive practices adopted 

0.917 

0.287 

1.546 

0 

0.000 

n/a 

n/a 

1 
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95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P-value 

Sample 

size 

Effect size calculation: 

SMD (standard calculation) 

0.789 

0.261 

1.317 

135 

0.519 

94.8% 

0.000 

8 

SMD (Cox adjusted 0R/RR) 

0.451 

-0.006 

0.907 

3031 

0.321 

99.8% 

0.000 

6 

Length of follow-up:* 

1 year or less 

2 years or less 

No obs. 
0.064 

-0.664 

0.191 

0 

0.000 

n/a 

n/a 

1 

More than 2 years 

0.577 

0.095 

1.059 

3057 

0.446 

99.8% 

0.000 

8 

Curriculum: 

IPM (no IPPM observations) 

0.740 

0.176 

1.304 

2803 

0.386 

99.9% 

0.000 

5 

Other curriculum 

0.777 

0.363 

1.190 

60 

0.219 

91.6% 

0.000 

6 

FFS+complementary 

0.196 

-0.041 

0.433 

9 

0.034 

77.5% 

0.012 

3 

Y ears of implementation of F F S programme: 

1 year or less 

0.654 

0.265 

1.042 

78 

0.136 

96.2% 

0.000 

4 

More than 1 year 

0.599 

0.254 

0.943 

984 

0.273 

99.1% 

0.000 

10 

Region: 

East Asia (EA) 

0.077 

0.066 

0.088 

0 

0.000 

n/a 

n/a 

1 

Latin America (LA) 

0.758 

0.429 

1.088 

14 

0.067 

85.3% 

0.001 

3 

Central Asia (CA) 

1.456 

1.184 

1.728 

0 

0.000 

n/a 

n/a 

1 

South Asia (SA) 

0.317 

-0.007 

0.642 

5 

0.050 

59.2% 

0.086 

3 

Sub-Saharan Africa (SSA) 

0.615 

-0.001 

1.230 

368 

0.546 

98.6% 

0.000 

6 

Crop: 

Rice 

0.795 

-0.523 

2.113 

26 

0.870 

96.2% 

0.000 

2 

Cotton 

0.708 

-0.240 

1.656 

877 

0.468 

99.9% 

0.000 

2 

Coffee, tea, cocoa 

0.386 

0.186 

0.586 

1 

0.002 

8.8% 

0.295 

2 

Other staples and vegetables 

0.602 

0.304 

0.901 

434 

0.147 

98.4% 

0.000 

8 


Note: * indicates data incomplete due to missing observations. 


Impacts on farmers' time use 

A small number of IPM-FFS studies also measured the burden of labour time (Birthal et al., 
2000; Tripp et al. 2005; Khan et al., 2007; Mandni &J iggins, 2008; Wu, 2010), estimating 
increases with adoption in India (Birthal et al., 2000; Mandni &J iggins, 2008). Birthal et al. 
(2000) noted that the increased time burden may be due to both time monitoring crops as 
well as harvesting bigger yields. The study found an increase of 25 per cent in labour time on 
IPM cotton farms as compared with farms using pestiddes, only about one- tenth of which 
was borne by women handpicking insect larvae, while one- third was due to time harvesting 
larger yields on FFS farms. However, the study was dassified as high risk of bias because the 
FFS had a larger average farm size and more irrigation than the comparison group, and these 
differences were not controlled in the analysis. Mandni and J iggins (2008) noted the burden 
of time for plant protection shifted towards women household members in India, and 
suggested therefore that availability of women might be a fador determining adoption of 
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IPM. Indeed, the relative costs of pesticide and labour may be important factors determining 
adoption. One study with medium risk of bias which found modest effects on outcomes 
noted pesticide costs were not a high share of variable production costs and dwarfed by costs 
of labour (Praneetvatakul &Waibel, 2006) . In this study, farmers with more farm area per 
household member were also more likely to drop out of field school training due to labour 
shortages and high opportunity costs of labour. 
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Agriculture outcomes (primary outcomes) 

Agricultural outcomes were measured in terms of yields (production per unit of land area 
cultivated, or its monetary value) or net revenues (value of production less cost per unit area 
ofland) (Appendix E). To take a few examples, Federetal. (2004), Rejesusetal. (2010), 
Pananurak (2010) and Wu (2010) measured production per unit ofland; Ali and Sharif 
(2012) measured total production; Davis et al. (2012) measured the monetary value of yields, 
which also captures output prices; Pananurak (2010) and Labarta (2005) measured net 
revenues, which capture both production but also (input and output) prices. We calculated 
the response ratio (RR) for agriculture outcomes, positive impacts being measured as values 
of RR significantly greater than 1. 

Impacts on yields 

Meta- analysis findings suggest that FFS training does on average lead to higher yields among 
FES graduates as compared with non- FFS farmers, both for I PM/ IPPM and other field 
school training (including other production techniques and/ or additional input or marketing 
support) (Figure 19). 

There appears to be substantial between- study heterogeneity (as measured by I-squared). 
Sensitivity analyses (Table 14) to explore possible sources of heterogeneity proceeded as per 
other outcomes. For IPM/ IPPM FFS participants, we found average effects to be small but 
statistically significant for medium risk of bias studies, representing an estimated increase of 
13 per cent in yields on average (RR=1.13, 95% Cl =1.04, 1.22; Q=53, Tau-sq=0.008, 1- 
sq=81%; 11 observations) (Figure 20). Similarly, we estimated average effects to be smaller 
for more rigorous quasi- experimental evaluation designs, and for outcomes measured in 
terms of yields as opposed to monetary measures of output. For outcomes measured at 
follow-up periods longer than two years ( excluding high- risk- of- bias studies), there is no 
evidence for statistically significant effects on yields. Moderator analysis suggests FFS have 
been relatively more successful in boosting yields of staples and vegetables in Africa using 
IPPM curriculum. Outcomes for FFS neighbours were fairly homogeneous across studies and 
analysis does not suggest outcomes are sensitive either to risk of bias or length of follow-up. 

We did not find evidence for spillovers to neighbour farmers living within FFS communities 
across all studies (RR=1.00, 95% Cl =0.98, 1.03; Q=13, Tau-sq=0.000, 1-sq=53%; 
observations=7) (Figure 19), when we exclude high- risk- of- bias studies (RR=1.02, 95% 

Cl =0.97, 1.08; Q=9, Tau-sq=0.002, 1-sq=66%; 4 observations) (Figure 20) or indeed for the 
majority of individual studies. 31 


31 The forest plot showing programme names rather than authors is in Appendix G. 
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Figure 19 Yields for IPM/ IPPM and other FFS farmers and IPM neighbours versus non-FFS 
comparison 


Study 

ID 

IPM/IPPM FFS 
Pananurak, 2010 (India) 

Labarta, 2005 (Nicaragua) 

Rejesus et al., 201 0 (Vietnam) 

Feder et al., 2004 (Indonesia) 

Wu, 2010 (China) 

Ali & Sharif, 201 1 (Pakistan) 

Yang et al., 2005 (China) 

Hiller et al., 2009 (Kenya) 

Khan et al., 2007 (Pakistan) 

Davis et al., 2012 (Tanzania) 

Pananurak, 2010 (Pakistan) 

Wandji et al., 2007 (Cameroon) 

Mutandwa & Mpangwa, 2004 (Zimbabwe) 
Zuger, 2004 (Peru) 

Carlberg et al., 2012 (Ghana) 

Yamazaki & Resosudarmo, 2008 (Indonesia) 
Van den Berg & Amarasinghe, 2002 (Sri Lanka) 
Davis et al., 201 2 (Kenya) 

Subtotal (l-squared = 82.2%, p = 0.000) 

Other FFS 

Van Rijn, 2010 (Peru) 

Naik et al., 2010 (India) 

Gockowski et al., 2010 (Ghana) 

Cavatassi et al., 201 1 (Ecuador) 

Birthal et al., 2000 (India) 

Dinpanah et al., 201 0 (Iran) 

Palis, 1998 (Philippines) 

Pande et al., 2009 (Nepal) 

Orozco Cirilo et al., 2008 (Mexico) 

Todo & Takahashi, 201 1 (Ethiopia) 

Subtotal (l-squared = 94.8%, p = 0.000) 

IPM neighbour farmers 
Pananurak, 2010 (India) 

Khan et al., 2007 (Pakistan) 

Feder et al., 2004 (Indonesia) 

Labarta, 2005 (Nicaragua) 

Wu, 2010 (China) 

Pananurak, 2010 (Pakistan) 

Yamazaki & Resosudarmo, 2008 (Indonesia) 
Subtotal (l-squared = 53.3%, p = 0.045) 



RR (95% Cl) 

0.80 

(0.61, 

1.05) 

0.97 

(0.92, 

1.02) 

0.97 

(0.72, 

1.31) 

0.98 

(0.96, 

1.01) 

1.08 

(1.03, 

1.14) 

1.09 

(1.03, 

1.15) 

1.15 

(0.94, 

1.41) 

1.17 

(0.53, 

2.56) 

1.17 

(0.97, 

1.42) 

1.23 

(1.00, 

1.51) 

1.24 

(1.01, 

1.54) 

1.32 

(1.07, 

1.63) 

1.36 

(1.06, 

1.73) 

1.44 

(1.09, 

1.92) 

1.58 

(1.19, 

2.10) 

1.67 

(1.23, 

2.26) 

1.68 

(1.30, 

2.18) 

1.81 

(1.15, 

2.84) 

1.16 

(1.09, 

1.24) 

0.86 

(0.63, 

1.18) 

0.89 

(0.83, 

0.96) 

1.14 

(1.03, 

1.25) 

1.22 

(0.97, 

1.53) 

1.24 

(1.13, 

1.36) 

1.32 

(1.22, 

1.42) 

1.36 

(0.97, 

1.92) 

2.11 

(1.25, 

3.56) 

2.62 

(2.23, 

3.08) 

2.71 

(1.11, 

6.60) 

1.34 

(1.10, 

1.63) 

0.79 

(0.63, 

1.00) 

0.97 

(0.74, 

1.26) 

0.99 

(0.97, 

1.01) 

1.00 

(0.99, 

1.01) 

1.03 

(0.99, 

1.08) 

1.03 

(0.86, 

1.25) 

1.43 

(1.05, 

1.96) 

1.00 

(0.98, 

1.03) 


.5 1 2 3 6 


Favours non-FFS 


Favours FFS participants 
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Figure 20 Yields for FFS participants and neighbours: excluding high- risk-of- bias studies 


Study 

ID RR (95% Cl) 


FFS participants 
Van Rijn, 2010 (Peru) 

Rejesus et al., 2010 (Vietnam) 

Feder et al., 2004 (Indonesia) 

Wu, 2010 (China) 

AN & Sharif, 201 1 (Pakistan) 

Cavatassi et al., 201 1 (Ecuador) 

Davis et al., 2012 (Tanzania) 

Pananurak, 2010 (Pakistan) 

Yamazaki & Resosudarmo, 2008 (Indonesia) 
Davis et al., 2012 (Kenya) 

Todo & Takahashi, 201 1 (Ethiopia) 

Subtotal (l-squared = 81.1%, p = 0.000) 


O 


0.86 (0.63, 1.18) 
0.97 (0.72, 1.31) 
0.98 (0.96, 1.01) 
1.08(1.03, 1.14) 
1.09(1.03, 1.15) 
1.22 (0.97, 1.53) 
1.23(1.00, 1.51) 
1.24(1.01, 1.54) 
1.67(1.23, 2.26) 
1.81 (1.15, 2.84) 
2.71 (1.11, 6.60) 
1.13(1.04, 1.22) 


FFS neighbours 

Feder et al., 2004 (Indonesia) 

Wu, 2010 (China) 

Pananurak, 2010 (Pakistan) 

Yamazaki & Resosudarmo, 2008 (Indonesia) 
Subtotal (l-squared = 65.9%, p = 0.032) 



0.99 (0.97, 1.01) 
1.03 (0.99, 1.08) 
1.03 (0.86, 1.25) 
1.43 (1.05, 1.96) 
1.02 (0.97, 1.08) 


.5 1 2 3 6 

Favours non-FFS Favours FFS participants 


Notably, as indicated in Table 14, we did not find significant effects for FFS participants (nor 
neighbours - results not reported) for the two programmes implemented at national scale 
(Indonesia and Vietnam National IPM Programmes) . Two studies replicated analysis of the 
Indonesian data (Feder et al., 2004 and Yamazaki & Resosudarmo, 2008), coming to 
different conclusions regarding effectiveness in terms of agricultural outcomes. Feder et al. 
(2004) found no effect of FFS on trained or exposed farmers' adoption of reduced pesticides 
or rice yields, mentioning poor implementation delivery as a possible explanation for poor 
performance (although not providing any evidence of this). They suggested scaling up of the 
programme may have had a negative effect on the average guality of trainers and their 
commitment to bottom- up approaches, given many of them may have been experienced 
extensionists trained to deliver using top-down methods in the past. Moreover, they noted 
that delays in transfers of funds to the field training organisers meant that FFS were not fully 
synchronised with the rice- growing season calendar and supplies of materials were irregular, 
and suggested that this may have had a negative impact on the quality of knowledge achieved 
by FFS graduates. 
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Table 14 Yields sensitivity analysis 



RR 

95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P- 

value 

Sample 

size 

1. FFS participants: all studies 

1.233 

1.145 

1.326 

315 

0.026 

91.4% 

0.000 

28 

Study design: 

"High quality" evaluation design 

1.098 

1.026 

1.175 

66 

0.007 

81.9% 

0.000 

13 

Cross-section regression 

1.238 

1.075 

1.426 

4 

0.002 

16.8% 

0.307 

4 

Longitudinal data (unadjusted analysis method) 

1.533 

0.917 

2.561 

44 

0.234 

93.2% 

0.000 

8 

Cross-section data (unadjusted analysis method) 

1.280 

1.069 

1.532 

77 

0.047 

92.2% 

0.000 

7 

Risk of bias: 

Medium risk of bias 

1.126 

1.040 

1.218 

53 

0.008 

81.1% 

0.000 

11 

Medium risk of bias (excluding Feder, Yamazaki) 

1.124 

1.046 

1.207 

15 

0.004 

47.6% 

0.054 

9 

Medium risk of bias (excluding Feder, Yamazaki, Rejesus) 

1.134 

1.052 

1.222 

15 

0.004 

52.2% 

0.041 

8 

High risk of bias 

1.294 

1.136 

1.474 

234 

0.060 

93.2% 

0.000 

17 

Outcome measure: 

Yields (production per unit of land) 

1.225 

1.128 

1.331 

285 

0.027 

92.3% 

0.000 

23 

Monetary value of production or sales* 

1.442 

1.172 

1.775 

50 

0.055 

89.9% 

0.000 

6 

Length of follow-up:* 

2 years or less 

1.173 

1.089 

1.264 

57 

0.012 

73.8% 

0.000 

16 

2 years or less (excl high risk of bias) 

1.235 

1.076 

1.417 

12 

0.014 

58.3% 

0.035 

6 

More than 2 years 

1.326 

1.053 

1.671 

184 

0.093 

96.2% 

0.000 

8 

More than 2 years (excl high risk of bias) 

1.068 

0.854 

1.336 

12 

0.037 

75.0% 

0.007 

4 

2. FFS neighbours: all studies 

1.003 

0.978 

1.028 

13 

0.000 

53.3% 

0.045 

7 

Risk of bias: 

Medium risk of bias 

1.022 

0.965 

1.082 

9 

0.002 

65.9% 

0.032 

4 

High risk of bias 

0.946 

0.828 

1.082 

4 

0.008 

49.1% 

0.140 

3 

Length of follow-up:* 

2 years or less 

1.007 

0.973 

1.043 

6 

0.001 

33.9% 

0.196 

5 

More than 2 years 

1.151 

0.804 

1.646 

5 

0.056 

81.7% 

0.019 

2 


Note: * indicates data incomplete due to missing observations. 


Yamazaki and Resosudarmo (2008) re-analysed the data from Feder et al. (2004), but 
additionally controlled for time exposed to FFS prior to FFS training, since FFS were carded 
out in the same villages for different fanners over a number of seasons, and time after 
graduation. This was in principle a sensible approach, given the authors' hypothesis about 
bigger short-term rather than longer-term effects, and they did find that allowing for time 
variables produced positive impacts of FFS training on rice yields in the short term. 
However, like Feder et al. (2004), they did not find any significant impact on adoption as 
measured by changes in pesticide costs (as reported in Figure 15). 32 We examined sensitivity 
of our findings by excluding these studies from analysis, in part because of potential 
spillovers suggested by Van den Berg and J iggins (2007) . Our results were insensitive to 
their exclusion (Table 14), as well as exclusion of studies with possibilities for other studies 


32 Yamazaki and Resosudarmo (2008) used a number of different specifications including a "spatial lag model" 
maximum likelihood model, which found significantly positive impacts on reducing pesticide costs, although this 
model was rejected in favour of the coefficients reported here on risk of bias grounds. 
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with possible spillovers (Rejesus et al., 2010) and active control groups (Davis et al., 2012 in 
Uganda) (see forest plots in Appendix G) . 

We conducted additional moderator analyses to assess whether findings vary by FFS 
curriculum, length of implementation, crop, region and farmer characteristics (Table 15). 

Due to the important policy relevance of the outcome, we examined whether findings 
differed when we excluded high- risk-of- bias studies, and we limit the following discussion to 
these results. We were able to identify significantly positive impacts among FFS graduates on 
yields for IPM-FFS training (9% increase; 6 observations) and FFS with complementary 
input or marketing support (20% increase; 3 observations), growing cotton (9% increase; 3 
observations) and other staples/ vegetables (37% increase; 4 observations). In addition, FFS 
which had been implemented for longer than two years experienced more significant effects 
( 17% increase; 6 observations) than those which had been operating for less (no significant 
difference; 5 observations) . However, there was no evidence for effects in programmes 
operating at national scale (no significant difference; 3 observations). We also found that 
only field schools which involved farmers of relatively high education levels exhibited 
significantly positive impacts on yields (9% increase; 3 studies), although we did not find any 
significant differences for landholdings. We were not able to identify significantly positive 
effects on average for any other sub-groups, due to the limited numbers of studies available. 

Consistent with the moderator analysis for adoption, we did not find significantly positive 
effects for rice FFS on yields. One study suggested that additional yield gain in 
technologically advanced rice production systems might be small and difficult to measure by 
recall surveys (Praneetvatakul &Waibel, 2006). In contrast, we estimated significant effects 
for cotton farmers in terms of both adoption and yields. Two of these studies based on 
evaluations of (genetically modified) Bt cotton programmes (Pananurak, 2010; Wu, 2010) 
found benefits of high yields and reduced pesticide use in incorporating improved seeds in 
the FFS curriculum. 


Table 15 Yields moderators analysis for FFS participants: excluding high- risk- of-bias studies 




95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P- 

value 

Sample 

size 

1. FFS -participants: all studies 

1.233 

1.145 

1.326 

315 

0.026 

91.4% 

0.000 

28 

Curriculum: 

IPM 

1.117 

1.045 

1.194 

72 

0.008 

83.3% 

0.000 

13 

IPPM 

1.373 

1.206 

1.563 

4 

0.000 

0.0% 

0.447 

5 

Other curriculum 

1.446 

0.895 

2.335 

159 

0.176 

98.7% 

0.000 

3 

FFS+complementary inputs/marketing support 

1.219 

1.074 

1.383 

14 

0.013 

57.8% 

0.027 

7 

Years of implementation of FFS programme: 

2 years or less 

1.308 

1.157 

1.479 

198 

0.051 

91.9% 

0.000 

17 

More than 2 years 

1.104 

1.024 

1.190 

51 

0.007 

80.3% 

0.000 

11 

Scale of implementation of FFS programme: 

Programme implemented at national scale 

1.142 

0.848 

1.536 

11 

0.055 

82% 

0.004 

3 

Pilot project or regional programme 

1.254 

1.151 

1.366 

250 

0.033 

90% 

0.000 

25 
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RR 

95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P- 

value 

Sample 

size 

Region: 

Central Asia (CA) 

1.318 

1.223 

1.421 

0 

0.000 

n/a 

n/a 

1 

East Asia (EA) 

1.103 

0.999 

1.217 

28 

0.008 

82.2% 

0.000 

6 

Latin America (LA) 

1.314 

0.838 

2.062 

141 

0.251 

97.2% 

0.000 

5 

South Asia (SA) 

1.155 

1.010 

1.321 

60 

0.027 

88.3% 

0.000 

8 

Sub-Saharan Africa (SSA) 

1.322 

1.168 

1.495 

12 

0.012 

44.0% 

0.000 

8 

Crop: 

Rice 

1.333 

1.094 

1.624 

89 

0.052 

93.3% 

0.000 

7 

Other staples & veg 

1.412 

1.113 

1.792 

180 

0.110 

95.5% 

0.000 

9 

Cotton 

1.133 

1.063 

1.208 

17 

0.004 

58.1% 

0.019 

8 

Permanent crops 

1.133 

0.975 

1.316 

5 

0.009 

39.4% 

0.175 

4 

FFS Farmer characteristics: 

Women farmers participated in FFS* 

1.308 

1.110 

1.542 

55 

0.039 

87% 

0.000 

8 

Women did not participate* 

F armer education years exceeds local average* 

No obs. 
1.180 

1.085 

1.283 

14 

0.006 

49.0% 

0.057 

8 

Farmer education years does not exceed local average* 

1.142 

0.967 

1.348 

26 

0.031 

81.1% 

0.000 

14 

Landholdings exceed local average* 

1.060 

0.877 

1.282 

19 

0.033 

78.6% 

0.001 

5 

Landholdings do not exceed local average* 

1.217 

1.105 

1.341 

9 

0.005 

32.4% 

0.181 

7 

FFS participants: excluding high-risk-of-bias studies 

1.126 

1.040 

1.218 

53 

0.008 

81.1% 

0.000 

11 

Curriculum: 

IPM 

1.090 

1.005 

1.182 

35 

0.006 

85.8% 

0.000 

6 

IPPM 

1.413 

0.981 

2.036 

2 

0.044 

57.8% 

0.124 

2 

Other curriculum 

FFS+complementary inputs/marketing support 

No obs. 
1.198 

0.793 

1.810 

7 

0.085 

71.5% 

0.030 

3 

Years of implementation of FFS programme: 

2 years or less 

1.084 

0.936 

1.256 

8 

0.012 

47.9% 

0.104 

5 

More than 2 years 

1.166 

1.045 

1.302 

37 

0.011 

86.6% 

0.000 

6 

Scale of implementation of FFS programme: 

Programme implemented at national scale 

1.142 

0.848 

1.536 

11 

0.055 

82% 

0.004 

3 

Pilot project or regional programme 

1.134 

1.052 

1.222 

15 

0.004 

52% 

0.041 

8 

Region: 

Central Asia (CA) 
East Asia (EA) 

No obs. 
1.073 

0.963 

1.196 

23 

0.007 

87.1% 

0.000 

4 

Latin America (LA) 

1.043 

0.744 

1.461 

3 

0.040 

67.6% 

0.079 

2 

South Asia (SA) 

1.117 

1.010 

1.235 

1 

0.003 

29.3% 

0.234 

2 

Sub-Saharan Africa (SSA) 

1.579 

1.057 

2.359 

5 

0.071 

58.5% 

0.090 

3 

Crop: 

Rice 

1.142 

0.848 

1.536 

2 

0.055 

82.3% 

0.004 

3 

Other staples and vegetables 

1.367 

1.098 

1.702 

5 

0.020 

43.6% 

0.150 

4 

Cotton 

1.091 

1.053 

1.130 

2 

0.000 

0.0% 

0.465 

3 

Permanent crops 

0.862 

0.631 

1.178 

0 

0.000 

n/a 

n/a 

1 

FFS farmer characteristics: 

Women farmers participated in FFS* 

1.251 

0.924 

1.693 

18 

0.074 

83.6% 

0.000 

4 


Women did not participate* No obs. 
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RR 

95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P- 

value 

Sample 

size 

Farmer education years exceeds local average* 

1.091 

1.029 

1.157 

2 

0.000 

4.5% 

0.351 

3 

Farmer education years does not exceed local average* 

1.220 

0.911 

1.632 

15 

0.055 

86.3% 

0.001 

3 

Landholdings exceed local average* 

1.131 

0.903 

1.415 

16 

0.040 

81.7% 

0.001 

4 

Landholdings do not exceed local average* 

1.247 

0.938 

1.658 

5 

0.039 

61.8% 

0.073 

3 


Note: * indicates data incomplete due to missing observations. 


The positive estimated effects of FFS for shorter-term studies (and conversely insignificant 
effects for longer-term studies) were found for both adoption and yields outcomes. However, 
most data are from indirect comparisons over programmes in different contexts with 
different follow-up periods. Several studies directly compared impacts over short and longer 
time periods among the same fanners. Praneetvatakul and Waibel (2006) reported impacts 
at one and three years after the intervention for FFS farmers in the Philippines, while 
Pananurak (2010) and Wu (2010) reported impacts at one and four- year follow-ups in 
China. Neither found significant differences in impacts for yields (or adoption) over these 
time periods, and both concluded FFS graduates retained their knowledge and continued 
applying IPM practices over time. However, more long-term impact evaluations are 
necessary to assess whether benefits are indeed sustained over time as these authors suggest. 

We also examined heterogeneity of diffusion from IPM field schools to non- participating 
neighbour farmers (Table 16) . The only studies which measured diffusion did so for 
programmes which had been implemented for more than two years. An initial positive 
diffusion effect found in China was not sustained (Wu, 2010), with both yield and knowledge 
gains in the neighbouring farmer groups diminishing considerably over time. 33 


Table 16 Yields moderator analysis for IPM- FFS neighbours 




95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P- 

value 

Sample 

size 

FFS neighbours: all studies 

1.003 

0.978 

1.028 

13 

0.000 

53.3% 

0.045 

7 

Years of implementation of FFS programme: 
2 years or less 
More than 2 years 

No obs. 
1.003 

0.978 

1.028 

13 

0.000 

53.3% 

0.045 

7 

Region: 









Central Asia (CA) 

No obs. 








East Asia (EA) 

1.023 

0.959 

1.092 

9 

0.002 

77% 

0.013 

3 

Latin America (LA) 

1.002 

0.992 

1.011 

0 

0.000 

n/a 

n/a 

1 

South Asia (SA) 

0.934 

0.794 

1.098 

3 

0.007 

36% 

0.210 

3 

Sub-Saharan Africa (SSA) 









Crop: 









Rice 

1.151 

0.804 

1.646 

5 

0.056 

82% 

0.019 

2 


33 Yamazaki and Resosudarmo (2008) also found initial productivity gains for rice yields in Indonesia diminished 
for both FFS graduates and neighbour fanners, as compared with non- FFS fanners. 
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RR 

95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P- 

value 

Sample 

size 

Other staples and vegetables 

1.002 

0.992 

1.011 

0 

0.000 

n/a 

n/a 

1 

Cotton 

0.987 

0.891 

1.094 

5 

0.005 

40% 

0.170 

4 

Permanent crops 

No obs. 








FFS farmer characteristics: 









Women farmers participated in FFS* 

1.151 

0.804 

1.646 

5 

0.056 

82% 

0.019 

2 

Women did not participate* 

No obs. 








FFS farmer education years exceeds local average* 

1.033 

0.990 

1.077 

0.2 

0.000 

0% 

0.886 

3 

FFS farmer education years does not exceed local average* 

1.013 

0.802 

1.278 

9 

0.032 

78% 

0.011 

3 

Landholdings exceed local average* 

1.013 

0.802 

1.278 

9 

0.032 

78% 

0.011 

3 

Landholdings do not exceed local average* 

1.012 

0.869 

1.178 

0.2 

0.000 

0% 

0.682 

2 


Note: * indicates data incomplete due to missing observations. 


In the final analysis of yields, we attempted to explain the heterogeneity using meta- 
regression (Table 17). Specification 1, which includes the full sample of 35 studies reporting 
yields effect sizes, co nfi rms much of the bivariate analysis, namely staples and vegetable 
crops tended to exhibit the biggest impacts, rigorous quasi- experimental studies tend to 
show significantly smaller impacts than other studies, and publication bias due to small 
study effects may be present (as indicated by Egger's regression test) . 


Table 17 Meta- regression analysis of agricultural yields 



■ (1) 

(2) 

(3) 

(4) 


Coeff* P>t 

Coeff* P>t 

Coeff* P>t 

Coeff* P>t 

l='Rigorous' quasi-exp study 

■0.155 0.071 

■0.225 0.026 

-0.014 0.866 


l=Neighbour fa rmers 

-0.083 0.410 

-0.083 0.445 

-0.081 0.244 


l=Rice crop 

0.159 0.104 

0.388 0.024 

0.389 0.019 

0.331 0.027 

1=0 ther staples/vegetables 

0.192 0.046 

0.245 0.026 

0.283 0.065 

0.307 0.040 

Length of follow-up (years) 


-0.054 0.119 

-0.088 0.040 

■0.077 0.053 

Average education (years) 



0.116 0.018 

0.116 0.009 

Log of standard error (Egger 





coefficient) 

1.280 0.028 

1.427 0.025 

1.204 0.102 

1.119 0.089 

Constant 

0.070 0.437 

0.187 0.146 

■0.652 0.055 

■0.677 0.019 

Numberof obs 

35 

31 

18 

18 

Tau-squared 

0.037 

0.039 

0.007 

0.004 

l-squared 

86.6% 

86.0% 

53.5% 

47.9% 

Adjusted R-squared 

30.1% 

33.1% 

45.3% 

69.8% 

Model F 

3.80 

3.39 

2.58 

3.31 

P rob > F 

0.009 

0.014 

0.085 

0.042 


Notes: coefficient estimates reported as natural logarithm. Bold indicates coefficient statistically significant at 10% level. 
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Specifications 2 adds follow-up period and specification 3 additionally includes average 
education of FFS- participants (both variables are measured in years); these specifications 
reduce the sample size due to missing observations. To conserve degrees of freedom, the 
fourth specification excludes variables which are not statistically significant in specification 
3. The analysis suggests impacts measured over longer periods were generally smaller while 
impacts were significantly bigger for more educated farmer participants. As per other 
outcome variables, years of education significantly increased the explanatory power of the 
model, suggesting field schools which targeted better educated farmers were also more 
effective in improving yields. In addition, an interaction between FFS- participant education 
and the neighbour farmer dummy variable is not statistically significant (results not 
reported), suggesting that field schools targeting better educated farmers may also have been 
more effective in promoting diffusion to neighbours. However, given the small sample size 
and relative instability of coefficients over specifications, these results should be interpreted 
cautiously. Until further evidence becomes available on the relationship between programme 
effects and characteristics of participating farmers such as education, income and social 
standing, we will not be able to conclude whether targeting farmers of higher socioeconomic 
status does lead to bigger increases in agricultural outcomes for FFS participants or 
neighbours. 

Impacts on net revenue 

It is possible that the production technologies promoted in FFS may lead to improved net 
revenues or profits (monetary value of production less costs), even where they do not 
improve yields, due to reduced reliance on pesticides and other purchased inputs. 34 We 
differentiated FFS programmes providing only training in I PM and those providing training 
in other (non-IPM/IPPM) curricula, from “FFS- plus" programmes which provided 
additional components of support for inputs and/ or marketing ( Figure 2 1) . 35 At least two 
points are worth noting here, the first being that the impacts on net revenues are larger in 
magnitude than yields, as we might also expect based on the bigger average impacts on 
pesticide expenditure than pesticide use. For the 'FFS- plus inputs/ marketing" farmer group 
in particular, we estimated revenues on average to be 150 per cent greater, although over a 
small sample size with wide dispersion in the pooled confidence interval (RR=2.57, 95% 

Cl =1.18, 5.58; 4 observations) and substantial estimated heterogeneity due to contextual 
factors (I-squared=96%, Tau-squared=0.56). For IPM-FFS participants, 36 revenues 
increased by a lesser amount. The results were sensitive to exclusion of high- risk- of- bias 
studies (Figure 22 and Table 18); for medium- risk-of- bias studies we estimated revenues to 
increase by 19 per cent on average (RR=1.19, 95% Q =1.11, 1.27; Q=l, Tau-sq=0, l-sq=0%; 2 
observations). 


34 Assuming, due to small farm size, any additional costs of labour are either met through in-kind contributions of 
family members or that costs of hired labour do not outweigh revenue gains. 

35 The forest plot showing programme names rather than authors is in Appendix G. 

36 We were not able to locate any studies of IPPM-FFS which measure revenues outcomes. 
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Figure 21 Net revenues for FFS farmers and neighbours versus non-FFS comparison 


Study Response ratio 

ID ES (95% Cl) 


IPM/IPPM FFS 

Labarta, 2005 (Nicaragua) < 

Pananurak, 2010 (India) 

Pananurak, 2010 (China) 

Pananurak, 2010 (Pakistan) 

Van den Berg et al., 2002 (Sri Lanka) 
Yang et al., 2005 (China) 

Khan et al., 2007 (Pakistan) 

Subtotal (l-squared = 69.9%, p = 0.003) 



0.28 (0.02, 3.48) 
1.06 (0.68, 1.66) 
1.17(1.08, 1.27) 
1.23 (1.09, 1.40) 
1.41 (1.19, 1.67) 
1.53 (1.10, 2.15) 
3.40 (1 .94, 5.97) 
1.34(1.15, 1.57) 


Other FFS curriculum 
Waarts et al., 2012 (Kenya) 

Naik et al., 2008 (India) 

Van de Fliert, 2000 (Indonesia) 
Subtotal (l-squared = 0.0%, p = 0.598) 



1.14(0.92, 1.41) 


1.25 (1.09, 1.42) 

-*■ 

1.31 (1.11, 1.55) 

❖ 

1.24(1.14, 1.37) 


FFS+input/marketing 
Birthal et al., 2000 (India) 

Van Rijn, 2010 (Peru) 

Cavatassi et al., 201 1 (Ecuador) 

Palis, 1998 (Philippines) 

Subtotal (l-squared = 96.2%, p = 0.000) 



1.43 (1.19, 1.72) 
2.00 (1.02, 3.94) 
3.34(1.56, 7.15) 
4.61 (3.83, 5.56) 
2.57(1.18,5.58) 


IPM neighbour farmers 
Pananurak, 2010 (India) 

Pananurak, 2010 (China) 

Pananurak, 2010 (Pakistan) 

Labarta, 2005 (Nicaragua) 

Khan et al., 2007 (Pakistan) 

Subtotal (l-squared = 0.0%, p = 0.706) 



0.93 (0.66, 1.32) 
1.07(1.00, 1.14) 
1.13 (1.01, 1.26) 
1.39 (0.66, 2.92) 
1.51 (0.51, 4.45) 
1.08 (1.03, 1.15) 


.2 .5123 

Favours non-FFS Favours FFS participants 


A second point of note is that the pooled effect size suggests that there may have been 
spillovers to non- participant neighbour farmers, although only partially in terms of 
magnitude of effect as compared with FFS graduates (RR=1.08, 95% Q =1.03, 1.15; Q=l, Tau- 
sq=0, l-sq=0%; 2 observations). However, the analysis excluding high- risk-of- bias studies 
(Figure 22) comprises only two interventions in China and Pakistan (Pananurak, 2010), both 
of which found positive effects in terms of revenues which were not reflected either in 
reductions in pesticide use or in improved yields, casting doubt on the credibility of the 
findings. In contrast, the increase in revenues for FFS participants in the same study was 
mirrored by improvements in both yields and pesticide reduction. 

The sensitivity and moderators analysis (Table 18) suggests that there may be differences in 
impacts on net revenues by crop, region and curriculum, but there were simply too few 
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rigorous studies to draw conclusions. 


Figure 22 Net revenues excluding high- risk-of- bias studies 


Study Response ratio 

ID ES (95% Cl) 


IPM-FFS farmers 



Pananurak, 2010 (China) 

• 

1.17 (1.08, 1.27) 

Pananurak, 201,0 (Pakistan) 

— 

1.23 (1.09, 1.40) 

Subtotal (l-squared = 0.0%, p = 0.461) 

0 

1.19 (1.11, 1.27) 

FFS+input/marketing 
Van Rijn, 2010 (Peru) 

Cavatassi et al., 201 1 (Ecuador) 
Subtotal (l-squared = 0.0%, p = 0.327) 


— 2.00 (1.02,3.94) 

• 3.34 (1.56,7.15) 

> 2.51(1.51,4.16) 

IPM neighbour farmers 



Pananurak, 2010 (China) 


1.07 (1.00, 1.14) 

Pananurak, 2010 (Pakistan) 

— 

1.13 (1.01, 1.26) 

Subtotal (l-squared = 0.0%, p = 0.424) 
1 

0 

1 r 

1.09 (1.03, 1.15) 


.5 1 2 3 

Favours non-FFS Favours FFS participants 


Table 18 Net revenues: sensitivity and moderator analyses 




95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P- 

value 

Sample 

size 

FFS participants: all studies 

1.282 

1.166 

1.410 

21 

0.011 

57% 

0.013 

10 

FFS participants: medium risk of bias 

1.187 

1.109 

1.270 

1 

0.000 

0% 

0.461 

2 

FFS participants: high risk of bias 

1.345 

1.166 

1.550 

17 

0.020 

59% 

0.017 

8 

FFS-plus inputs/marketing participants: all studies 

2.569 

1.184 

5.576 

78 

0.560 

96% 

0.000 

4 

FFS-plus participants: medium risk of bias 

2.507 

1.513 

4.155 

1 

0.000 

0% 

0.327 

2 

FFS-plus participants: high risk of bias 

2.568 

0.815 

8.093 

77 

0.677 

99% 

0.000 

2 

FFS participants: evaluation design 









Rigorous quasi-experimental study 

1.183 

1.106 

1.265 

2 

0.000 

0% 

0.567 

4 

Cross-section regression (adjusted analysis) 

1.243 

1.088 

1.419 

1 

0.000 

3% 

0.311 

2 

Longitudinal data (unadjusted analysis) 

3.400 

1.937 

5.967 

0 

0.000 

n/a 

n/a 

1 

Cross-section data (unadjusted analysismethod) 

1.324 

1.196 

1.465 

2 

0.000 

3% 

0.357 

3 


FFS participants: length of follow-up 
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95% confidence interval 

Q 

Tau- 

sq 

l-sq 

P- 

value 

Sample 

size 

2 years or less 

1.284 

1.100 

1.499 

18 

0.021 

66% 

0.007 

7 

More than 2 years 

1.306 

1.196 

1.426 

1 

0.000 

0% 

0.538 

3 

FFS participants: curriculum 

IPM 

1.343 

1.147 

1.572 

20 

0.023 

70% 

0.003 

7 

IPPM 

Other curriculum 

No obs. 
1.245 

1.135 

1.365 

1 

0.000 

0% 

0.598 

3 

FFS participants: years of implementation of programme 

2 years or less 

1.293 

1.195 

1.399 

4 

0.000 

0% 

0.459 

5 

More than 2 years 

1.303 

1.053 

1.610 

15 

0.029 

74% 

0.004 

5 

FFS participants: region 

Central Asia (CA) 
East Asia (EA) 

No obs. 
1.251 

1.104 

1.418 

4 

0.006 

44% 

0.169 

3 

Latin America (LA) 

0.280 

0.023 

3.481 

0 

0.000 

n/a 

n/a 

1 

South Asia (SA) 

1.358 

1.143 

1.613 

14 

0.023 

71% 

0.008 

5 

Sub-Saharan Africa (SSA) 

1.141 

0.925 

1.409 

0 

0.000 

n/a 

n/a 

1 

FFS participants: crop 

Rice 

1.408 

1.187 

1.670 

0 

0.000 

n/a 

n/a 

1 

Other staples and vegetables 

1.267 

1.144 

1.404 

2 

0.000 

0% 

0.449 

3 

Cotton 

1.347 

1.114 

1.629 

16 

0.027 

75% 

0.003 

5 

Cotton (excl high risk of bias) 

1.187 

1.109 

1.270 

1 

0.000 

0% 

0.461 

2 

Permanent crops 

1.141 

0.925 

1.409 

0 

0.000 

n/a 

n/a 

1 

FFS neighbours: all studies 

1.084 

1.026 

1.145 

2 

0.000 

0% 

0.706 

5 

FFS neighbours: medium risk of bias 

1.086 

1.027 

1.148 

1 

0.000 

0% 

0.424 

2 

FFS neighbours: high risk of bias 

1.034 

0.766 

1.396 

1 

0.000 

0% 

0.491 

3 


Finally, we examined heterogeneity of revenues using meta- regression (Table 19) . The meta- 
regression indicated that FFS which were provided alongside inputs and/ or marketing 
support lead to significantly better outcomes, when controlling for crop type, study design, 
and fanner. Although heterogeneity remained high (I-sq=78%), the small sample size limited 
our ability to explore other sources of differences across studies. However, no evidence was 
found for publication bias or for significant effects for other variables such as FFS- 
participant education (results not reported). 


Table 19 Meta- regression analysis of net revenues 



Coeff* 

t-statistic 

P>t 

l='Rigorous' quasi-exp study 

-0.098 

-0.52 

0.608 

l=Neighbour fa rmers 

-0.076 

-0.36 

0.726 

l=Rice crop 

0.385 

1.57 

0.138 

l=FFS+input/marketing 

0.557 

2.68 

0.018 

Constant 

0.269 

2.10 

0.054 


Number of obs 
Tau-squared 
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'•squared 77i8% 

Adjusted R-squared 5g g 0 / 0 

M° de|F 456 

Prob >F 0014 

Notes: coefficient estimates reported as natural logarithm. Bold indicates coefficient statistically significant at 10% level. 


109 



Other final outcomes (secondary outcomes) 

Effects on environment 

Five studies of I PM/ 1 PPM fanner field schools reported effects on environmental outcomes; 
in three cases we were able to calculate effect sizes and standard errors (Appendix E) . One 
study measured positive effects on soil fertility of IPPM-FFS among FFS participants in 
Uganda (RR=1. 79, 95% Cl =1.15, 2.78; Friis-Hansenetal., 2004). Four further studies 
measured impacts of five I PM- FFS interventions on environmental outcomes using the 
environmental impact quotient (EIQ) score (Kovach et al., 1992), which estimates changes in 
outcomes indirectly based on reported reductions in chemical pesticide use. 37 The findings of 
the meta- analysis for EIQ scores are suggestive of benefits to FFS graduates (Figure 23). 
Walter- Echols and Soornro (2005) also estimated reductions in EIQ in India (RR=0.68) and 
Pakistan (RR=0.58) but did not report information to calculate standard errors so are not 
included in the meta- analysis. 

EIQ is the only outcome for which estimated effects on non- participant neighbour farmers 
were significantly positive across all studies regardless of risk of bias. However, given the 
EIQ is estimated from reported pesticide use, for which we did not find any impacts on 
neighbour farmers, it seems unlikely that these significant effects on average are common. 
Indeed, when we excluded high- risk- of- bias studies from the analysis (including results for 
neighbour farmers from Cavatassi et al., 20 11, for which we have approximated standard 
errors) the results for FFS participants remained significant (RR=0.61, 95% Cl =0.48, 0.78; 
Q=3, Tau-sq=0.01, 1-sq=33%; 3 observations), while those for neighbours were no longer 
statistically significant (RR=0. 70, 95% Cl =0.43, 1.14; Q=0.5, Tau-sq=0.00, l-sq=0%; 2 
observations) (Figure 24). 


37 Pananurak (2010) noted that EIQ scores were not calculated for China "due to lack of information about 
pesticide compounds" (p. 69). 


no 



Figure 23 Environment outcomes: environmental impact quotient (EIQ) 


Study Response ratio 

ID ES (95% Cl) 


FFS participants 


Pananurak, 2010 (India) * 

0.52 (0.32, 0.85) 

Praneetvatakul &' Waibel, 2006 (Thailand) — ■ — 

0.54 (0.39, 0.76) 

Pananurak, 2010 (Pakistan) — ■ — 

0.55 (0.41,0.75) 

Cavatassi et at, 201 1 (Ecuador) ■- 

— 0.82 (0.55,1.23) 

Subtotal (l-squared = 8.0%, p = 0.353) 

0.59 (0.49, 0.71) 

FFS neighbours 


Pananurak, 2010 (India) ■ 

0.58(0.24,1.41) 

Pananurak, 2010 (Pakistan) ■ 

0.64(0.37,1.10) 

Cavatassi et at, 201 1 (Ecuador) H — 

0.69 (0.43, 1.11) 

Praneetvatakul & Waibel, 2006 (Thailand) 

♦ 1 .04 (0.32, 3.40) 

Subtotal (l-squared = 0.0%, p = 0.878) 
1 1 1 

0.68 (0.49, 0.93) 
T 


.1 .2 .512 

Favours FFS participants Favours non-FFS 
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Figure 24 Environmental impact quotient (EIQ) excluding high-risk-of-bias studies 


Study 

ID 


Response ratio 
ES (95% Cl) 


FFS participants 


Praneetvatakul & Waibel, 2006 (Thailand) ■ — 

0.54 (0.39, 0.76) 

Pananurak, 2010 (Pakistan) ♦ 

0.55 (0.41, 0.75) 

Cavatassi et al., 201 1 (Ecuador)) ■— 

— 0.82(0.55,1.23) 

Subtotal (l-squared = 32.6%, p = 0.227) 

0.61 (0.48, 0.77) 

FFS neighbours 


Pananurak, 2010 (Pakistan) ■ 

0.64(0.37,1.10) 

Praneetvatakul & Waibel, 2006 (Thailand) 

♦ 1 .04 (0.32, 3.40) 

Subtotal (l-squared = 0.0%, p = 0.462) 
T T 1 

> 0.70(0.42,1.14) 

1 


.1 .2 .512 

Favours FFS participants Favours non-FFS 


Effects on health 

Health outcomes were measured in four studies, all of which were assessed as being of high 
risk of bias. We were able to calculate effect estimates and confidence intervals in two studies 
(Amera, 2009; Labarta, 2005) (Appendix E). 38 One study in Nicaragua compared self- 
reported health outcomes among FFS participants with those of non- participants (Labarta, 
2005) . The results, shown in Figure 25, suggest - counter-intuitively - that FFS beneficiaries 
experienced greater respiratory difficulties, though cases of eye irritation, stomach ache and 
blurred vision were not significantly different. In addition, neighbour farmers experienced 
greater eye irritation than non-FFS comparison farmers. However, the specifications used 
also found that the length of time after graduation from FFS significantly reduced the 
incidence of respiratory difficulties. 


38 DANIDA (2011) estimated reductions in health expenditure (RR=0.19) but did not report information to 
calculate standard errors. Zuger (2004) estimated increases in use of protective clothing and gloves during 
pesticide preparation and spraying measures of adoption of practices which may affect health outcomes. 
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Figure 25 Health outcomes in Nicaragua (Labarta, 2005) 


Study 

ID ES (95% Cl) 


FFS participants 

Lack of respiratory difficulties 

Lack of eye irritation - 

Lack of stomach-ache 



0.19 (0.04, 0.85) 
0.40 (0.15, 1.03) 
0.96 (0.71, 1.29) 

Lack of blurred vision 

— 

- 

1.00 (0.81, 1.24) 

FFS neighbours 
Lack of eye irritation 
Lack of stomach-ache 



0.63 (0.42, 0.93) 
1.07 (0.91, 1.25) 

Lack of respiratory difficulties 
Lack of blurred vision 

1 

1 r 

1.29 (0.72, 2.31) 
1.55 (0.74, 3.24) 


.5 12 5 

Favours non-FFS Favours FFS participants 


It appears that adoption of practices was a problem in Labarta (2005), as the authors 
indicated that 5 of the 13 FFS did not include a conventional "farmer practice" (control) plot; 
of the 8 that did, half observed lower yields in the IPM plot compared with the control plot, 
while 6 observed lower net revenue in the IPM plot compared with the conventional plot. In 
these cases, when comparative trials of IPM found higher revenues or yields in the IPM plot 
relative to the conventional plot, fanners were more likely to adopt the IPM practices 
included in the curriculum. A second study (Amera, 2009) reported perverse results in terms 
of pesticide poisoning in Kenya (RR=1. 10, 95% C3 =0.93, 1.30), despite significant estimated 
reductions in pesticide use (RR=0. 61, 95% Cl =0.52, 0.71). 

It is difficult therefore not to conclude from these studies that reverse causality is driving the 
counter-intuitive outcomes, where farmers who most recently participated did so because of 
an existing respiratory problem for which analysis was unable to account. 39 

Effects on empowerment 

A total of four studies collected quantitative data on some measure of empowerment 


39 The effect sizes calculated from Labarta used in this analysis are from standard regression analysis (ordered 
probit) without control for possible endogeneity; Haussman tests were performed and the authors were unable to 
reject the null hypothesis of exogeneity of FFS participation. 
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(Appendix E) . Some outcomes were reported more dearly than others, and in only one case 
were we able to calculate confidence intervals (Van Rijn, 20 10) . Thus, Rusike et al. (2004) 
reported "empowerment attitudes" and Hiller et al. (2009) measured change over time in 
farmers' perceptions of empowerment; further details on the definition of empowerment 
were not reported. In contrast, Friis- Hansen and Duveskog (2012) reported empowerment 
indices for innovation uptake, access to services, engaging with markets and collective 
action/ sotial relations, from which we calculated a pooled point estimate. In the case of Van 
Rijn (20 10), we calculated a pooled variable measuring the probability that farmers felt 
improvements in self-esteem, including feeling capable of solving problems in the field, 
feeling comfortable in giving an opinion, and partidpatingin the community (RR=2.13, 95% 
Q =1.46, 3.12). Figure 26 shows the forest plot for empowerment outcomes, including point 
estimates only for those studies where confidence intervals could not be calculated. The 
limited evidence suggests benefidal impacts, which is supported by qualitative evidence 
reported in Chapter 5, although further quantitative studies are needed to support the 
validity findings. 

A study of the IFAD-FAO East African FFS programme examined impacts on production by 
gender (Davis et al., 20 12) . The study showed high rates of FFS partidpation by women, who 
comprised two- thirds of field school partidpants in Kenya and half of those in Uganda. 

While women did not seem to have problems attending the FFS, female FFS partidpants did 
not benefit significantly more than female non- partidpants in both of these countries in 
terms of improved income. While women form a large part of the agricultural labour force, 
they may not be the household or community dedsion- makers and therefore the agents of 
change in adoption of practices. However, women FFS partidpants did benefit more than 
women non- partidpants in Tanzania, where women's partidpation made up one- third of the 
FFS programme intake. Davis et al. (2012) also found that increases in productivity for 
partidpants with no formal education were greater than for any other group of farmers, 
suggesting the experiential learning and demonstration focus of the FFS appears to allow 
low- literacy farmers to activdy partidpate and learn. 
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Figure 26 Empowerment outcomes (includes studies without standard error estimates) 
Study 


ID 

Response ratio 

Hiller et al., 2009 (Kenya) 

■ 

Friis-Flansen & Duveskog, 2012 (Uganda) 

♦ 

Friis-Flansen & Duveskog, 2012 (Tanzania) 

m 

Friis-Flansen & Duveskog, 2012 (Kenya) 

♦ 

Van Rijn, 2010 (Peru) 

Rusike et al., 2004 (Zimbabwe) 

1 1 

♦ 

1 1 


Favours non-FFS Favours FFS participants 
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5 Results of Qualitative Synthesis 


5.1 SEARCH RESULTS 


In this section we report the results of the synthesis of qualitative studies addressing review 
question (2) on the barriers to and enablers of FFS effectiveness. The included studies do not 
necessarily report on the same interventions as the included effectiveness studies. 

Figure 27 provides a detailed outline of the search strategy and review process for the studies 
included to address our second review question. The search for qualitative studies was 
implemented in parallel with the search for evidence on effectiveness. Over a thousand 
( 1, 112) abstracts were systematically screened to assess whether full texts of the papers 
should be obtained for the qualitative synthesis component of the review according to the 
separate inclusion criteria for studies answering review question (2); 314 papers were 
retrieved for full text analysis. Two independent researchers systematically reviewed the 
texts and assessed whether the papers should be included for qualitative synthesis. 
Disagreements were resolved by discussion. Most (252) of these studies were excluded on 
relevance (they did not report on FFS nor provide evidence on barriers to and enablers of 
FFS effectiveness), or methodology (they were not primary research or failed to meet the 
methodology criteria as set out in the study selection criteria) . Twenty studies 
(corresponding to 27 articles) were included in the analysis. 
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Figure 27 Qualitative synthesis search results 



Title screening 
criteria: 

Developing country 
FFS 

Primary research 


Abstract screening 
criteria: 

Developing country 
FFS 

Primary research 
Country of analysis 


47 not available 


Full text inclusion 
criteria: 

FFS primary research 
Qualitative or 
descriptive studies 
Barriers/enablers 
Methods reporting 


257 studies excluded: 

123 excl on methodology 
127 excl on relevance 
7 excl as duplicates 




751 studies excluded: 

85 excl on methodology 
274 excl on relevance 
392 excl as duplicates 


v y 


V 


20 studies (27 articles) 
included in qualitative 
synthesis 


5.2 STUDY CHARACTERISTICS 


Table 20 displays the basic characteristics of the included studies. Eleven of the studies were 
conducted in Africa (Cameroon, Kenya, Liberia, Uganda, Tanzania and Zimbabwe), seven of 
the studies were conducted in Asia (Bangladesh, Cambodia, Indonesia, the Philippines and 
India) and two in Latin America (Honduras, and Trinidad and Tobago). All the studies were 
addressing issues pertaining to FFS, including process and implementation, knowledge 
formation and farmers' experiences of FFS participation. 

Where multiple studies reported evidence from the same country, the data were collected 
from different FFS in different districts. For example, two studies reported on evidence from 
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Indonesia. However, one study reported evidence from Central J ava province (Van de Fliert, 
1993), while the other reported evidence from the West J ava province (Winarto, 2004) . 
Similarly, there were five studies that reported evidence from Kenya. Of these, three of the 
studies collected data from different districts (Friis- Hansen et al., 2012; Machacha, 2008; 
Najjar, 2009), and two studies collected data from different FFS programmes (Hiller et al., 
2009; Karanj a- Lumumba et al., 2007). Finally, there were two studies reporting evidence 
from the Philippines, each reporting on a different FFS programme (Palis, 2002; Rola and 
Baril, 1997). 

The studies differed in the methods they used, the quality of reporting and the 
implementation of these methods. As is described in more detail in Appendix C, we assessed 
the quality of the included studies using a predetermined checklist. Full results of the quality 
appraisal are reported in Appendix F, and Figure 28 provides a summary of the quality 
assessment for each included study according to each criterion in our quality appraisal 
checklist. 

A large number of the studies suffered from weaknesses in reporting, making quality 
appraisal more challenging. Of particular concern was the limited reporting on sampling and 
sample characteristics, clarity of analysis and lack of presentation of data to support findings. 

All the papers dearly stated their research aim and provided contextual background. The 
vast majority also contained a description of appropriate partidpant selection procedures 
(sampling) and provided details of sample characteristics such as sample size and location. 
Most studies also included adequate details of how data were collected, but only around one- 
third set out how data had been recorded. Furthermore, only 35 per cent of the studies 
provided a clear and explidt explanation of how data were analysed. Nearly all the studies 
(85%) situated the research within the relevant literature or set out a clear theoretical 
framework. The methodology was judged appropriate for the research question with the 
author^ s) offering a dear justification for their approach in 50 per cent of the cases and doing 
so partially in 45 per cent of the cases; 5 per cent of the studies did not provide clear enough 
information to make a judgment regarding the appropriateness of the methodology. Criteria 
for appropriate sampling induded providing an explanation of why the selected partidpants 
were suitable to provide the knowledge sought by the study and whether there were any 
issues around recruitment such as some potential partidpants choosing not to take part. 
Eight of the twenty studies addressed all these issues, ten did so partially and two others did 
not provide clear enough information for the criterion to be properly assessed. 

The "appropriate methods of data collection" criterion required studies to use appropriate 
and justified methods, including that data be collected in a way that addressed the research 
issue at hand and for the researcher to discuss saturation of data. Sixty per cent of the 
studies partially satisfied these requirements and the remaining 40 per cent fully satisfied 
them. The "appropriate analysis" criterion required studies to provide the data that 
supported reported findings, and required that the relationship between researcher and 
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participants had been considered, and judged the extent to which contradictory data were 
taken into account. Fifteen per cent of the studies fulfilled these requirements, 55 per cent 
did so partially, 20 per cent did not satisfy the criteria at all and 10 per cent were not 
sufficiently dear to make a judgment. Two of the most common weaknesses were the failure 
to consider the relationship between researcher and partidpant and the effect this might 
have on the data collected, and the absence of any real consideration of the significance of 
contrary evidence or explanations. Another common shortcoming of the included studies 
was incomplete reporting of data supporting the findings. 

All the induded studies provided at least some evidence of having triangulated their results, 
with 15 per cent of studies satisfying one of the following requirements and the remaining 85 
per cent satisfying two or more of them: the verification of findings using two or more data 
sources; the application of multiple methods; employment of multiple investigators; 
investigation of multiple theories. However, none of the studies employed theoretical 
triangulation approaches. To satisfy the "darity of analysis and conclusions" criterion, 
researchers needed to have done all of the following: discussed the credibility of their 
findings; discussed evidence both for and against their own arguments; made their findings 
explidt and discussed them in relation to the original research question. Twenty- five per 
cent of studies satisfied all these requirements, 65 per cent did so partially while 10 per cent 
failed to address them adequately. Finally, only small proportions of the papers clearly set 
out any ethical considerations (25%) or addressed potential conflicts of interest ( 15%) in 
terms of author relationships with funder or implementer. 
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Table 20 Summary characteristics of included studies 


Study Study objectives 


Country FFS programme 


Methods of data collection Methods of analysis Summary of quality assessment 


DAN IDA (2011) The aim of the study was "to analyse and 
FFS document- in a gender perspective - the 

results and the lessons learned from using 
the FFS approach in the ASPS II in 
Bangladesh" 

"According to the ToR the Evaluation 
should, in particular, provide information 
about whether and to which extent the FFS 
approach is contributing to increased 
income and food security at household 
level, as well as to women's involvement in 
development processes in Bangladesh." 


David, 2007 'The aim of the paper is to examine what 
knowledge and skills farmers acquire in 
FFS, what they transmit to non-participants 
and the social impacts of this training 
approach." 


Bangladesh The Bangladesh Agricultural 

Extension Component (AEC), part 
of the Agricultural Sector Program 
Support (ASPS) II 


Cameroon Sustainable Tree Tops Programme, 
hosted by the International Institute 
of Tropical Agriculture 


Focus group discussions (FGDs) 
and/or individual interviews with 
extension workers, farmer 
organisation members, NGOs, 
private service providers and 
traders, key stakeholders at village 
level (FFS facilitators/trainers, FFS 
(former) participants and control 
groups), direct observation of FFS 
sessions, FFS 

technologies/activities implemented 
by FFS graduates and training-of- 
trainer learning sessions. An 
evaluation matrix with key 
evaluation questions and indicators 
was used to prepare standardised 
checklists for the FGDs with 
different stakeholder groups (FFS 
farmers, control farmers, 
facilitators/trainers, community- 
based organisation/ Farmer Club 
leaders etc.) to ensure that 
similar type of data and information 
would be collected across the 
components and geographic 

areas. 

N on-structured interviews with 
selected FFS graduates, formal 
survey of FFS participants and non- 
participants, interviews with FFS 
graduates, farmers from non-FFS 
villages and knowledge recipients. 
Interviews conducted in French in 
most cases, sometimes in the local 
language if necessary. The survey 
instrument for FFS and non-FFS 
farmers covered questions about 
uptake of practices/knowledge 
learned in FFS, diffusion of 
knowledge acquired from FFS to 
household and non-household 
members, method of diffusion, 


The study describes the methods of 
analysis used for the quantitative 
component (ex postPSM or DID 
PSM but with recall data) but does 
not describe in detail the methods 
used to synthesise findings from the 
qualitative component. Context 
analysis and programme theory 
based evaluation thinking were 
parts of the overall analytical 
framework. Not much detail of how 
this was operationalised though. 


The authors do not explicitly report 
how the data were analysed, but it 
is obvious from the study that they 
used descriptive statistics and t- 
tests. 


This is a mixed methods study, with 
a propensity score matching 
component and a qualitative 
section. The major limitations of the 
study are the lack of reporting of 
some of the data in support of the 
findings presented; failure to 
consider the relationship between 
participants and researchers and 
lack of discussion on contradictory 
findings/arguments. 


Overall the study suffers from 
limitations in reporting, analysis and 
transparency. It is mainly focused 
on assessing impact on knowledge 
and diffusion, for which the 
methodology is not appropriate. In 
addition, there might be potential for 
conflict of interest: The author 
seems to be an employee of the 
International Institute of Tropical 
Agriculture which hosted the 
intervention. 

Information on barriers and 
enablers is somewhat limited. 
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Methods of data collection 


Study 


Country FFS programme 


Methods of analysis Summary of quality assessment 


Dolly, 2009 
(incorporates 
Dolly, 2005, 
2008, 2009) 


The study set out the following objective: T rinidad 

"to assess ... 14 FFS in Trinidad and and 

T obago from the perspective of the T obago 

Simpson and Owens (2002) key elements 
facing extension programs and FFS. These 
are: relevancy and responsiveness of FFS 
to local concerns, systems learning and the 
generation of new knowledge, information 
flow and farmer-to-farmer communication, 
institutionalisation and local organisational 
development, changes in relationships, and 
the integration of the FFS into existing 
programs." (Dolly, 2009) 


F riis-Hansen, 
2008 

(incorporates 
F riis-Hansen 
2005 and F riis- 
Hansen etal., 
2004) 


'This study is inspired by the development- Uganda 
research approach to impact assessment 
and aims to (1) place the agricultural 
technology development among poor 
farmers in Soroti district, Uganda in a 
socioeconomic and institutional context, 
and (2) differentiate between different well- 
being categories when assessing the 
impact of access to improved technologies, 
farmer empowerment, and access to 
privatised demand-driven advisory 
services." 


Initiatives by the Commonwealth 
Agricultural Bureau International 
and Trinidad and Tobago's Ministry 
of Agriculture Lands and Marine 
Resources 


social impacts of FFS and a test to 
assess knowledge related to four 
broad areas covered in FFS: cocoa 
physiology, disease and pest 
management, rational pesticide use 

and post-harvest operations. 

Interviews of the whole sample No information provided 

conducted by the researcher, 

focused interviews with six 

volunteer participants. Observation 

of field days, committee meetings 

(of the committee of the Extension, 

Training and Information Services of 
the MALMR (including study and 
recording of minutes of 12 monthly 
meetings)), and observations of 
activities of the implementing 
agency (Caura Valley Farmers 
Association, CVFA). Document 
analysis - the researcher studied 
the minutes of 12 monthly meetings. 


Global Integrated Pest Management 
Facility Project under the auspices 
of the FAO 


'The study is based on two sets of 
field work in 2001 and 2004." (Friis 
Hansen etal., 2004, p. 251) 

Two quantitative surveys (2001, 
2004); interviews; strengths, 
weaknesses, opportunities and 
threats; participatory rural appraisal; 
qualitative in-depth life history 
interviews. For the 2004 survey 
wellbeing indicators were developed 
based on consultation with farmers, 
and poverty index was developed 
based on this exercise. 


Method of analysis for quantitative 
survey is reported. They used 
SPSS statistics software for the 
analysis, computing correlation and 
chi squared tests between group 
membership and poverty indicators. 

It is not clear how the in-depth 
interviews were analysed. Not clear 
how information on changing 
opportunity structures, 
responsiveness of advisory services 
was analysed. 


Overall the study suffers from 
limitations in reporting, analysis and 
transparency. Major shortcomings 
of the study include incomplete 
reporting on sample characteristics, 
lack of reporting on methods of 
analysis and limited analysis. The 
authors do not present data to 
support their findings, do not 
discuss the relationship between 
the researcher and participants, do 
not present/ discuss contradictory 
evidence and do not discuss the 
credibility of their findings. The lack 
of information about the analysis 
and no presentation of data make it 
difficult to assess the reliability of 
the findings. 

Information on barriers and 
enablers is somewhat limited. 

The study does not clearly report on 
methods of sample selection and 
methods analysis. The main 
shortcoming is the lack of clarity 
and depth of analysis. Another 
major limitation is that not all 
findings are supported by a 
presentation of evidence and the 
authors arrive at contradictory 
findings between publications. 


Not clear how information on 
changing opportunity structures, 
responsiveness of advisory services 
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and other institutional context were 
collected. 

Friis-Hansen, 

2011,2012 

The study objectives were: "(1) to explore 
how FFS impacts on the daily lives of 
participants, their family relationships and 
their relations with other members of the 
group and the wider community; (2) to 
examine the relationship between collective 
processes and gender relations; (3) to 
explore transformative learning as a 
possible explanation for the learning and 
changes that individuals experience from 
participating in FFS." 

Kenya 

IFAD-FAO Programme in East 
Africa 

Individual and group interviews. 
Observations and key informant 
interviews. The in-depth interviews 
followed an interview guide 
developed to ensure that certain 
questions were covered. 

Constant comparative approach. 
'The data were separated from the 
original transcript using NVIVO- 
QSR (version 8) in order to identify 
their essential elements. Codings of 
responses (Miles & Huberman, 
1994) were made in an inductive 
manner, where themes were 
developed based on emerging 
similarities of expression. As a 
result, common themes were 
identified and grouped into main 
and sub-themes (Lincoln & Guba, 
1985)." 

The main shortcomings are a lack 
of clarity on sampling procedures 
(especially for the key informant 
interviews) and some lack of 
transparency and clarity in the 
analysis. The study does not 
present data on contradictory 
evidence so it is not clear to what 
extent it has been taken into 
account. 

Gottret & 
Cordoba, 2004 

The paper aimed to tackle the following 
questions: the role of social actors on 
innovation processes; the ways in which 
institutions influence actors' access to 
knowledge, technology, information and the 
complementary resources necessary for 
innovation; the influence of external 
intervention, particularly that of the state, on 
innovation processes; the impact of 

Honduras 

The Central American Integrated 
Pest Management Programme or 
Programa de Manejo Integrado de 
Plagas en America Central 
(PROMIPAC) 

Document review, in-depth 
interviews with government 
stakeholders, participation in 
PROMIPAC workshops, in-depth 
interviews with FFS facilitators, an 
FFS farmer and a non-participant 
farmer. 

Limited description of analysis, 
although detailed description of the 
analytical framework which was 
used for the analysis. The paper 
states: 'The information collected 
through the review of documents 
and in-depth interviews was 
analysed using the logical 
framework presented in Figure 1." . 

The study is relatively clear, but 
there is a lack of detail on the 
methods of analysis and only limited 
presentation of data. The authors 
do not discuss the relationship 
between researchers and 
participants and the credibility of 
their findings. 


innovation on strategies to generate 
livelihoods and their sustainability; ways in 
which the state can facilitate and promote 
innovation processes with small-scale 
producers more effectively. 


Hiller et al., The objectives of the study were to: Kenya The Kenya Tea Development 

2009 "compare the sustainability score of FFS Agency (KTDA)/Lipton Sustainable 

farmers at the start and after graduation; to Agriculture Project 

assess the impact of the FFS on 
knowledge, implementation of good 
agricultural practices, and livelihood 
aspects before and after FFS participation 
and between FFS and non FFS 
participants; to assess the perception of 

farmers of the FFS approach." 


Semi-structured individual 
questionnaire for FFS members; 
semi-structured individual 
questionnaire for non FFS 
members. Data collected by student 
enumerators originating from the 
area. 

of analysis and lack of engagement 
with the relevant literature. Although 

not entirely clear, the authors 


No information provided The study has detailed reporting on 

data collection and a clear analysis 
and conclusion, clearly stating the 
limitations of drawing causal 
inferences from the findings. The 
main shortcomings of the study 
include lack of reDortina on methods 
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Hofisi, 2003 The aim of the study was to: Zimbabwe 

"a) examine the relevance of a FFS 
learning approach to the resource poor 
farmers of the Zambezi Valley, b) To 
examine the impact of active 
experimentation and Indigenous 
Knowledge on the farmers' learning 
process, c) To find out problems which are 
being experienced in the FFS and their 
possible solutions, d) To see whether the 

approach may have a wider application." 

Isubikalu, 2007 To "describe and analyze this FFS-based Uganda 
process of agro-technological adaptation, in 
order to arrive at some overall judgment 
about how well the FFS-based 'innovation 
system' is working." 


Karanja- The main purpose of the study was "to Kenya 

Lumumba et al., establish the role of emerging second order 
2007 associations in sustaining and providing 

continuity to community based groups by: 

(1) Understanding the key elements in the 
design of the second order associations; (2) 

Identifying services provided by second 
order organizations to community-based 
organizations and farmers; (3) 

Understanding the role of second-order 
associations in strengthening and 
sustaining a two-way flow of information 
between the farmers and collaborators and 
(4) Identifying the financing mechanisms 


Zambezi Valley Organic Cotton 
Project 


Mapping (groups); semi-structured 
interviews (groups and individuals); 
matrix scoring, score ranking and 
historical trend analysis conducted. 
Paper provides brief descriptions of 
how these methods were 
implemented (p. 23). 


Very little detail is provided on how 
the collected data were integrated 
and analysed. The author notes that 
the participants were involved in the 
analysis of the findings. 


conducting the evaluation (follow-up 
survey) appear to be independent of 
the implementing and funding 

agencies. 

This is a well-conducted study with 
some shortcomings, including 
incomplete reporting on sampling 
and methods of analysis. In 
addition, the author notes that the 
researcher has been involved in 
training and the overall 
implementation of the project, but 
does not consider the potential 
conflict of interest arising from this. 


a. Integrated Production and Pest 
Management project/scheme 
(IPPM) implemented by FAO 

b. Integrated Pest Management 
(IPM) project implemented by 
Makerere University (MAK) 

c. Integrated Soil Productivity 
Improvement (IS PI) under Africa 
2000 network (A2N) 

d. Integrated Production and Post- 
harvest Handling Management 
(IPPHM) implemented by 
International Potato Centre (CIP). 
Safe Pesticide Use and Handling 
(SPUH) implemented by MAK 
The FFS networks arose 
independently from the East African 
Sub-Regional Pilot Project for 
Farmers Field Schools, funded by 
IFAD and implemented by the FAO. 


Participant observation and direct 
observation, prolonged stay in the 
field, including working with the 
farmers, key informants, interviews, 
photography, focus groups, group 
discussions, attendance at formal 
meetings. Participant ranking used 
to identify local priorities. 


"Pre-tested checklists were used to 
guide the Focus Group Discussions 
and the in-depth interviews." 

(p. 1346) 

No further detail provided. 


Transcribed data from the field were 
reduced and sorted into emerging 
themes and patterns to describe 
and explain processes and activities 
in FFS, using the context- 
mechanism outcomes framework of 
interactions in F F S . To convey a 
flavour of the feelings of the 
different respondents, some 
excerpts based on their own 
wording have been built into the 
analysis as direct quotes. 


No explicit mention of how data 
were analysed. It is not clear from 
the text what analyses were 
conducted and how information was 
extracted and combined. 


Overall, this is a well-designed 
study with clear reporting on data 
collection and analysis. Main 
shortcomings include the failure to 
present and consider 
alternative/contradictory evidence. 
The study could also have 
presented more data as it is difficult 
to judge how representative the 
information provided is. 


The study has several major 
limitations. The study does not 
clearly report on the sampling 
characteristics and data collection. 
The major shortcoming of this paper 
is that there are no data presented 
and no explicit description of how 
the analysis was conducted. It is 
unclear how the information from in- 
depth interviews and focus groups 
was combined and how the authors 
arrived at the findings presented. It 
is not clear whether the 
methodology was appropriate to 
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the Second Order Associations use to 
ensure they are self-sustaining." 





answer the research questions 
presented. 

Machacha, 

2008 

The first objective is to learn more about 
Farmer Field Schools as organized in this 
region today; the second objective is to 
compare some elements among the FFS in 
order to identify the characteristics and 
processes that are associated with 
outcomes that provide members the 
greatest benefits in terms of learning and 
sustainable livelihoods. 

Kenya 

Ongoing programmes, initially set 
up by the FAO and then by the 
Kenyan Ministry of Agriculture 
Extension department and the 
Kenya Agricultural Research 
Institute 

Interviews and direct observation of 
farmers' fields. F ocus groups. The 
researcher explained the purposes 
of the study once to the entire group 
through reading the prepared 
Information Letter and responding 
to any questions that arose. The 
purposes of the study were 
explained again to any person 
selected for an individual interview 
through reading the prepared 
Information Letter and responding 
to any questions they had. 

The only explicit description of 
method of analysis is that data were 
analysed qualitatively. The data are 
then presented thematically in the 
findings section, prefaced with a 
sentence saying so. 

Overall, this is a well-designed 
study with clear reporting and 
analysis. It adopts a case study 
design and various methods of data 
collection appropriate for the 
research questions. Minor 
shortcomings of this study are 
incomplete reporting on data 
analysis methods, lack of reporting 
on data recording methods and 
failure to discuss the relationship 
between the researcher and 
participants and other limitations of 
the study. 

Mancini etal., 
2007 

'The aim of the study was to test the S L 
[sustainable livelihoods] framework to 
document the emic perception of change in 
people's livelihoods since they attended 
IPMFFS (or overthe same time period 
without IP M F F S for the control groups)." 

India 

EU-FAO Cotton IPM, and Indian 
Government 

Questionnaire. Spider diagramming, 
respondents were first elicited to 
define the different types of capital 
during an interview with a trained 
facilitator, an outline diagram was 
drawn and participants were asked 
to rate the state of these capitals at 
present and 2 years ago. 

Sustainable livelihoods framework 
and statistical analysis. The initial 
process of generating and 
interpreting findings was 
participatory. 

The study has clear reporting on 
data collection and analysis. A 
major shortcoming is that the 
majority of the relevant findings (on 
barriers and enablers) are not 
supported by the presentation of 
data. Another limitation is that the 
conclusions seem at times to 
overstate/misrepresent the 
presented findings. In addition, 
there is a potential conflict of 
interest as the study has been 
funded by the implementing agency. 
The funding source is 
acknowledged, but the conflict of 
interest is not disclosed. 

Najjar, 2009 

'This research explores transformative 
learning occurring in FFS in Kenya 
following five specific objectives: (1) to 
understand the characteristics of the local 
agricultural production systems; (2) to 
consider gender specific interests with 
implications for the FFS program; (3) to 
assess the conditions of learning; (4) to 
understand the individual learning 
outcomes of farmers involved in the FFS; 
(5) to determine whether the extension 
activities promote broader community 

Kenya 

Ministry of Agriculture 

Unstructured and semi-structured 
interviews, focus group discussions, 
gender and social analysis, Rapid 
Rural Appraisal (RRA), Participatory 
Rural Appraisal (PRA), Resource 
Flow Maps (RFM), Farm Transects 
(FT), transect walks, local NGO 
workshops on gender sensitisation 
and a document review of farmers' 
notes, NGO and government FFS 
forms and relevant minutes of 
meetings. Participant observation 

Gender and social analysis using an 
adaptation of the Moser framework. 
QSR NVivo, a qualitative data- 
analysis software, was used to code 
and explore data in search of 
themes and regularities. Excel and 
Arc Map were used for representing 
the gender specific adoption rates 
of FFS technologies in the Mwora 
FFS and the case study area 
location, respectively. Data were 
coded into 100 nodes derived from 

Overall, this is a well-designed case 
study with very good reporting and 
analysis. The biggest limitation of 
the study is a lack of information on 
the sample characteristics at the 
individual level. 
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thinking about sustainable agriculture." 



(which included participating in farm 
work, attending FFS sessions and 
farm visits), (p. 6) 

the interview questions. These 
nodes were then grouped into five 
data sets, description of agriculture 
system, gender issues, how 
learning occurred, what was learned 
and social action and change, 
representing the five research 
objectives. 


Palis 2002 
(incorporates 
thesis and 2006, 
as well as 
several co- 
authored: Palis 
etal.,2002, 
2005) 

The study aimed to: 

"a) Understand the process ofIPM adoption 
among FFS farmers; 

b) Identify the effective sources of social 
capital that facilitates farmer sharing, 
learning and practice ofIPM; 

c) Assess the building up of social capital 
and its impact on the promotion ofIPM by 
understanding the process of farmer 
sharing and learning, and the spread of 
IPM practice; 

d) Assess the impact of IPM, mediated by 
social capital, both at the individual and 
village level; and 

e) Utilize the identified effective sources of 
social capital in coming up with a strategy 
in upscaling IPM in a swift, efficient, and 
spontaneous manner." 

Philippines 

The Barangay (village) Integrated 
Pest Management (BIPM) project 

A range of different methods of data 
collection used; household survey 
using semi-structured 
questionnaire; observations (regular 
periodic collection of the input- 
output data was done every season, 
from 1993 until 1995 and 1999). 
Semi-structured questionnaire; 
focus group discussion; 
participatory mapping; and key 
informant interviews; social network 
survey; household map; genealogy. 

Household map, farm map, 
genealogy, and case presentations 
in supporting qualitative inferences. 
For quantitative inferences, used 
graphical analysis and basic 
statistical tests like t-test, F-test, 
frequency distributions, chi-square 
test, Fisher's exact test, and logistic 
regression. 

The study has clear reporting and 
analysis of rich data. Minor 
shortcomings include failure to 
consider the researcher- participant 
relationship in the analysis and the 
potential for conflict of interest as 
the author notes: "1 was involved 
with this project throughout the 
study period" (Palis, 2006) and then 
returned to conduct dissertation 
research. 

Pedersen etal., 
2008 

'To assess institutional capacity, identify 
implementation successes and bottlenecks, 
quality, and sustainability ... The objective 
was to make a status, survey, review, 
analyse, discuss, and conclude on district 
implemented FFS in Mbeya region 
targeting all those who were 
DADS/supported." 

Tanzania 

District Agricultural Development 
Support (DADS), supported by local 
training institute. Further, support 
was allocated to rural district 
farmers by IFAD and the Ministry of 
Agriculture. FAO and WWF 
supported Mbarali district. Extensive 
support also provided by DANIDA in 
seven districts. 

Semi-structured interviews, field 
visits to the FFS groups and 
physical field visits/registrations. 

A mixture of descriptive statistics, 
correlations and content analysis. 

The study has several major 
limitations. It does not clearly report 
on the data collection methods and 
does not report on the methods of 
analysis. Moreover, it does not link 
to existing literature, nor present all 
the data in support for its findings 
and conclusions. It is therefore 
difficult to assess how reliable the 
findings of this study are. 

Rola and Baril, 
1997 

This is a case study of "the economic, 
social and educational impact of FFS. 
Analysis included a) measuring knowledge 
gains of FFS farmers, b) monitoring 
changes in practices, c) investigating the 
economics of these changed practices, d) 
estimating cost savings due to higher 
efficiencies of FFS graduates." 

Philippines 

Philippine National IPM Programme 

Recall benchmark survey, plot-size 
measurement, weekly monitoring of 
crop level data, in-house FFS 
documentation, weekly AESA data, 
postproduction household data, soil 
samples, FFS evaluation, 
knowledge tests, information 
network survey, and decision- 

No information provided 

The study has several major 
limitations, with limited information 
on barriers and enablers. The major 
shortcomings of this study include 
incomplete reporting on sampling, 
context, lack of reporting on data 
recording and methods of analysis, 
failure to present data for the 
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making variables. 


Simpson, 1997 The study set out to answer the following Cambodia 
research question: "Has the IPM-FFS 
extension mode succeeded in empowering 
farmers to promote the long term 
sustainability of agriculture in SvayTeap. 

The thesis focuses on analysing IP M 
extension experiences from a village level 
perspective ... The goal is to shed some 
light on the project's success and illuminate 
areas which may need to be improved." 


Van de F liert, 'To describe and analyse the processes Indonesia 
1993 and effects taking place at the village level 

as a result of IP M training and 
dissemination, and to provide useful 
management information to the programme 
for further implementation" including 
"information with respect to characteristics 
and diversity of intended and actual 
programme beneficiaries and to actual 
achievements considering behaviour of 
farmers and farm level effects as a result of 


National IPM Programme ofMAFF 
(Ministry of Agriculture, Forests and 
Fisheries) 


The National IPM Programme, 
implemented as a government 
programme through the existing 
extension system (assisted by FAO 
and expatriate and local experts) 


Key informant interviews (mostly 
with international NGOs, 
academics, government workers 
and members of the Cambodian 
national IPM team, sessions often 
very informal, conducted in French 
or English, used forgetting 
information, bouncing ideas and 
triangulating information, also 
conducted with village leaders); 
semi-structured interviews 
(informed by the key informant 
interviews, open ended, avoiding 
leading questions, while planned to 
be with individuals often joined by 
the rest of the family, lasted 1-3 
hours) and focus group discussions 
(with FFS and non-FFS participants, 
and also with more vulnerable 
groups, mainly facilitated by the 
translator); participant observations/ 
daily life (participation by the 
researcher in the daily village life). 
Focus groups, individual interviews, 
qualitative observations, 
quantitative surveys, farmers' 
record keeping, field observations. 


Apart from saying this is a case 
study analysis, no information 
provided. 


Case study design: multi-case 
design evaluating IPM training at 
village level. 


implementation (only relevant 
section) aspect of the study, failure 
to present and consider evidence 
against the authors' arguments and 
failure to consider the researcher- 
participant relationship. In addition, 
there is a potential conflict of 
interest as the research has been 
funded by an FFS-implementing 

agency. 

The study has clear reporting and 
analysis with rich information on 
barriers and enablers. The major 
shortcomings include incomplete 
reporting on sampling and methods 
of analysis, incomplete presentation 
of data on which the findings are 
based and failure to present/ 
discuss evidence contradictory to 
the author's arguments. 


Overall a strong study with rich data 
and analysis. The study could have 
been clearer in reporting of the 
analysis methods. The main 
shortcoming is that little of the data 
from which the findings and 
conclusions are drawn is presented 
in the study. 
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IPM training and dissemination" 

Van DerWiele, 
2004 

The study set out to examine the following: 
"Whether and how external assistance, 
within the field of community and ecological 
design specifically, can be used more 
effectively to enable smallholder 
households to secure their basic needs, 
promote self-reliance, and adopt 
sustainable resource management 
practices as a means of breaking the 
unending cycle of environmental 
degradation and persistent poverty." 

Liberia 

United Methodist Church 
Committee on Reliefs Sustainable 
Agriculture and Rural Development 
Initiative 

In-depth interviews, semi-structured 
interviews, field observations, 
conversations with key informants 
and reviews of official documents, 
photographic documentation of field 
observations. Detailed description 
of how interviews were conducted 
provided, including interview 
questions. 

Grounded theory analysis 
approach. Analysis conducted using 
an iterative process - the constant 
comparative method. Involves 
creating a data matrix for each 
respondent, comparison of 
emerging patterns within and 
between respondents, using content 
analysis. 

Overall this is a strong study with 
only minor limitations. It adopts a 
case study design and various 
methods of data collection 
appropriate for the research 
questions. The reporting on 
methods is extensive and 
transparent. Data are presented 
and the analysis and conclusions 
are clearly presented. The main 
shortcoming is incomplete reporting 
on sampling procedures. 

Winarto, 2004 

'The main objectives of this book are not 
only to examine the interaction between 
those with differing modes of knowledge 
acquisition, but to investigate the 
subsequent processes in which knowledge 
is continuously formed and changed, and 

Indonesia 

The National IPM Programme, 
implemented as a government 
programme through the existing 
extension system (assisted by FAO 
and expatriate and local experts) 

Direct observation, semi-structured/ 
informal interviews and 
questionnaires. 

Not reported. The author used a 
comparative case study design, 
though presents findings from only 
one of the locations. 

The study contains rich material, but 
there is some lack of clarity on 
methods used, including a lack of 
clarity in reporting on sampling, data 
collection and failure to report on 
methods of analysis. 


the extent to which action plays a part in 
the continuous knowledge formation ... The 
study will examine the following issues: ... 
how the learning process took place; how 
the new knowledge was disseminated and 
adopted by other farmers; to what extent 
those ideas were diversely or uniformly 
adopted and which elements of previous 
knowledge persisted." 


Figure 28 Summary of critical appraisal across all included studies 
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5.3 SYNTHESIS RESULTS 


This section reports the results of the synthesis of findings from the qualitative 
studies, presented using the hypothesised programme theory as an overall 
framework for structuring the synthesis. Based on the findings from the synthesis 
we also present an updated programme theory (Figure 29) . The figure includes 
revised assumptions as well as statements about the barriers and enablers related to 
each step in the causal chain, based on the evidence in the included studies. 

As can be seen from the critical appraisal (Figure 28), there are considerable 
weaknesses in the underlying evidence base. The findings should therefore be 
interpreted with caution and the findings presented below should be considered 
suggestive rather than definitive. There were likely biases in terms of what is 
studied. For instance, a lot of the qualitative evidence related to barriers, rather than 
enablers. 

Programme components 

General inputs 

As with any programme FFS require a range of different inputs at the outset in order 
to establish the programme and start delivering services to participants. There was a 
lack of systematic reporting of the resources devoted to individual FFS interventions 
and assessment of the quality of the services delivered to farmers in the included 
studies. However, some studies reported a lack of (timely) funding and inadequate 
provision of resources, including inputs and finances, as potential reasons for 
implementation failures (Pedersen et al., 2008; Najjar, 2009; Winarto, 2004). These 
included insufficient and incidental provision of inputs, discrepancies between 
budgeted and received payments, payment delays and difficulties with logistics and 
dissemination. In addition, selection of inappropriate FFS sites and a lack of 
adequate follow-up and support during pest outbreaks were noted as constraints. 

Facilitators 

The facilitator is an important input in FFS programmes, and according to the 
programme theory adequate training of facilitators is a key assumption for the FFS 
training to lead to improved knowledge and skills among participating farmers. If 
facilitators are not adequately trained the quality of the training received by farmers 
may be lower, affecting knowledge formation, adoption and final outcomes. 

Six of the seven studies that reported on training of facilitators suggested that there 
were issues related to a lack of appropriate training, resources and ongoing support 
of facilitators (DANIDA, 2011; Gottret & Cordoba, 2004; Hofisi, 2003; Pedersen et 
al., 2008; Rola &Baril, 1997; Simpson, 1997). Examples of deficiencies included 
gaps in the curriculum covered in the training of facilitators and lack of focus on 
participatory techniques and facilitation skills. The studies also highlighted issues 
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related to a lack of ongoing support or backstopping and lack of inputs to support 
facilitators in their roles. 

One theme which was not included in the original programme theory relates to the 
selection and characteristics of FFS facilitators. Four studies suggested appropriate 
criteria for selecting facilitators were important for identifying candidates suitable to 
be facilitators (DANIDA, 2011; Hofisi, 2003; Machacha, 2008; Winarto, 2004). 
These studies found that rather than high levels of education, characteristics such as 
personal attitude, maturity, literacy, leadership skills and experience with farming 
might be more important for facilitators to perform their role successfully. 
Facilitators not having the right characteristics from the outset appeared to have 
influenced their ability to successfully perform the role of FFS facilitator. 

Moreover, a large difference between farmers and facilitators in terms of 
socioeconomic characteristics may prevent farmers from fully participating and 
making suggestions or raising concerns ( Simpson, 1997) . Finally, the gender of the 
facilitator might be important, in particular in more conservative contexts. Some 
studies found that women preferred female facilitators (Hofisi, 2003; Van Der 
Wiele, 2004), suggesting that FFS programmes aiming to target women farmers 
should take account of this when selecting facilitators. 

The characteristics of the facilitators and the training they receive in turn effect 
another theme which was highlighted by several of the included studies: the 
relationship between facilitators and participants. Three of the studies suggested an 
imbalance in the farmer- facilitator relationship as a potential barrier to farmers' 
learning and adoption (Isubikalu, 2007; Najjar, 2009; Simpson, 1997), with farmers 
not feeling comfortable enough to admit when they did not understand something, 
or to voice concerns or make suggestions. 

Targeting criteria and procedures 

While not included in the original programme theory our synthesis suggests a key 
assumption in the FFS programme theory may be that the right farmers are targeted 
and reached by the FFS. The targeted farmers must be willing and able to participate 
in FFS training throughout the full season, and be able to implement FFS practices 
in their fields. Those studies that provided information on targeting, selection 
procedures and group composition, have been linked to FFS effectiveness analysis 
above, and are covered in a separate piece on FFS targeting (Phillips et al., 20 14) . 

Available evidence suggests that structural factors such as socioeconomic status, 
gender and cultural norms influence both who is targeted and who is able to 
participate in FFS. In some cases targeting procedures privileged the elite and more 
affluent (DANIDA, 2011; Simpson, 1997; Van deFliert, 1993), or excluded women 
and the poor (DANIDA, 2011; Najjar, 2009; Simpson, 1997). In other cases poverty 
and ill health prevented farmers from participating (Gottret & Cordoba, 2004; 
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Najj ar, 2009) . In yet another case, an absence of targeting criteria was suggested as 
a barrier to effectiveness (Pedersen et al., 2008). 

While several studies found that women were able to participate in FFS (Hofisi, 
2003; Machacha, 2008; Najjar, 2009; Rola&Baril, 1997), other studies found that 
women were often not allowed to participate by their husbands or could not 
participate due to time constraints and lack of access to resources and land 
(DANIDA, 2011; Hofisi, 2003; Najjar, 2009; VandeFliert, 1993, VanDerWiele, 
2004). 

Farmers' education levels and motivation to participate was another key 
determining factor highlighted by existing evidence (Gottret & Cordoba, 2004, 
Hofisi, 2003). The most commonly cited reason to join a field school was to improve 
knowledge, skills and livelihoods (DANIDA, 2011; Friis- Hansen, 2008; Hofisi, 2003, 
Najjar, 2009). Several studies found high levels of drop-out (Friis- Hansen, 2008; 
Gottret & Cordoba, 2004; Machacha, 2008; Najjar, 2009; Rola&Baril, 1997; 
Winarto, 2004). Unmet expectations of hand-outs or loans were the most commonly 
cited explanation in the literature ( Friis- Hansen, 2008; Hofisi, 2003; Machacha, 
2008; Najjar, 2009). Additional reasons for drop-outs included lack of interest, 
access and time (Gottet &Cordoba, 2004; Machacha, 2008; Najjar, 2009). 

Overall, the review identified several key barriers related to targeting, such as 
inappropriate selection criteria and targeting procedures, and structural barriers to 
participation such as gender, poverty and cultural norms. If there is not a considered 
approach to targeting, this may result in farmers participating for the wrong reasons 
and ultimately dropping out. Alternatively, beneficiaries may not have the right 
characteristics, such as education levels, or access to land and resources, to be able 
to fully benefit from the FFS training. FFS should target farmers with appropriate 
education levels, motivation and access to land, and those who do not live too far 
away to attend the weekly FFS sessions. 

Approach to learning 

A key characteristic of FFS design is that programmes should follow a participatory 
approach, based on theories of adult education and discovery- based learning 
(Pontius et al., 2002). A participatory, bottom- up approach to the training is 
considered important in communicating complex concepts, strengthening problem- 
solving capabilities and ultimately convincing farmers to adopt the practices 
promoted in FFS. 

Over half of the included studies included themes pertaining to the way in which 
FFS were delivered in the field (DANIDA, 2011; Dolly, 2009; Friis- Hansen, 2008; 
Gottret &Cordoba, 2004; Hofisi, 2003; Isubikalu, 2007; Machacha, 2008; Najjar, 
2009; Pedersen et al., 2008; Rola &Baril, 1997; Van de Fliert, 1993, Winarto, 2004). 
Some of the studies represented cases where FFS were delivered in a top-down 
manner, using a transfer of technologies approach rather than according to the 
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original approach characterising FFS (Isubikalu, 2007; Pedersen et al., 2008; 

Najjar, 2009; Simpson, 1997). The top-down approach seemed to have occurred at 
different levels - a bias towards talk, rather than practice, experiments being 
managed by the facilitators rather than farmers, or the delivery of the FFS in terms 
of the overall approach, budgeting, monitoring and inputs provided. 

Nevertheless, a greater proportion of the included studies represented cases where 
FFS were delivered in a participatory manner (DANIDA, 2011; Dolly, 2009; Friis- 
Hansen, 2008; Gottret & Cordoba, 2004; Hofisi, 2003; Machacha, 2008; Rola & 
Baril, 1997; Van de Fliert, 1993; Winarto, 2004), 40 with farmers taking part in field 
trials and development of the curriculum. According to the participating farmers of 
one FFS, their active experimentation and information- sharing enhanced their 
learning and increased knowledge and ownership of the resulting farming systems. 

Related to this, there is consistent evidence that due to the complexity of the 
curriculum, observability is important for farmers to trust the messages promoted in 
FFS and to develop analytical skills (Dolly, 2009; Hofisi, 2oo3; Machacha, 2008; 
Palis, 2002; Simpson, 1997; VanDerWiele, 2004; Winarto, 2004). The 
observability of IPM practices and their relative advantage compared with 
conventional methods was found to be important in enhancing farmers' confidence 
and trust in IPM messages. Similarly, in a few cases facilitators were unable to 
demonstrate observable benefits from FFS practices and this acted as a barrier to 
adoption. 

The extent to which the FFS in the included studies were delivered according to the 
original principles upon which the intervention was designed varied, ranging from 
approaches to learning similar to traditional extension, to "true FFS". Where 
implementation in practice was characterised by a more top-down, transfer of 
technology approach farmers were not able to see and practice IPM, and this 
appeared to have been a barrier to farmers developing their knowledge and 
analytical skills. 

The evidence highlighted the importance of empirical study, successful 
demonstration plots, and visibility of benefits in other farmers' plots in convincing 
farmers to adopt new practices. The apparent importance of observability and 
relative advantage of FFS practices in farmers' adoption is consistent with Rogers' 
theory of diffusion (2003), which suggests peoples' perceptions of these 
characteristics are among the important attributes determining adoption of 
innovation. 


40 The description and analysis of actual implementation of FFS is limited in both Friis- Hansen (2008) 
and Friis- Hansen et al. (20 12) . They provide what is a rather generic description of an "ideal type” FFS, 
backed up with very limited description and no data on implementation. 
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FFS content and coverage 

The FFS curriculum is another major component of the intervention. For farmers to 
learn and adopt the practices promoted in FFS, an assumption is that the content of 
the curriculum is relevant to the needs of farmers, appropriate for the local context 
in terms of the crops it covers and feasible for farmers to implement in their fields. 
Most of the included studies reported themes relating to the FFS curriculum 
(DANIDA, 2011; David, 2007; Dolly, 2009; Gottret& Cordoba, 2004; Hiller etal., 
2009; Hofisi, 2003; Isubikalu, 2007; Mandni etal., 2007; Najjar, 2009; Palis, 

2002; Simpson, 1997; Van de Fliert, 1993; Van Der Wide, 2004). 

From the studies it appears that in most cases the FFS curriculum was relevant and 
appropriate to the local context (DANIDA, 2011; Dolly, 2009; Gottret & Cordoba, 
2004; Hiller et al., 2009; Hofisi, 2003; Mandni et al., 2007; Van de Fliert, 1993), for 
instance by responding to local concerns over economic and environmental costs of 
pestiddes (Dolly, 2009; Mandni et al., 2007). 

However, in some cases FFS curricula appeared to lack relevance to the local context 
(Isubikalu, 2007; Najjar, 2009; Simpson, 1997), and the major barrier in these cases 
seemed to be that the curriculum was not suffidently tailored to the needs and 
resources of the farmers it was targeting. For instance, in some cases the practices 
were seen to be too labour- or time- intensive (Isubikalu, 2007) or failed to address 
the issues that were of highest priority among fanners ( Simpson, 1997) . 

Farmers may have a range of concerns, such as water management, fertilisation, 
diversification and marketing. A failure to incorporate a broader range of concerns 
in the curriculum was perceived by farmers to be a weakness in several studies 
(DANIDA, 2011; Mandni et al., 2007; Najjar, 2009; Van de Fliert, 1993, Winarto, 
2004), who expressed that the curriculum was not suffidently comprehensive in its 
coverage. Therefore a broader focus might encourage partidpation in FFS, and could 
potentially also fadlitate greater improvement of agricultural outcomes. 
Nevertheless, as one case from Kenya demonstrates (Pedersen et al., 2008), trying 
to cover too many topics or crops in one cycle may put the technical quality of the 
intervention at risk. 

Limited evidence suggests that in some cases the complexity of IPM made it difficult 
for farmers to implement all the IPM practices on their crops (Gottret & Cordoba, 
2004; Van de Fliert, 1993; Winarto, 2004). Some of the analytical tools used were 
too complex and perceived as impractical in relation to the time, energy and 
resources they required (Winarto, 2004) . For instance, the use of forms to record 
field sampling with formulae to calculate percentages for damages and prevalence of 
insects was found to be of little practical use for the farmers, who abandoned this in 
favour of simply recording what they observed in their fields (Van de Fliert, 1993) . 
On the other hand, some studies found that including indigenous and practical 
knowledge, rather than theoretical concepts, may enhance farmers' understanding 
of FFS practices (David, 2007; Hofisi, 2003). 
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Related to the complexity of the aimculum is the discourse and language used for 
communication in FFS. An assumption not included in our original programme 
theory, but which emerged from the synthesis is that the curriculum needs to be 
communicated in a language farmers understand. Evidence from Indonesia 
suggested that when facilitators used unfamiliar foreign and scientific terms which 
were part of their vocabulary, such as "economic threshold level" (ETL) and 
"ecosystem", these terms were not easily understood by farmers (Van de Fliert, 

1993; Winarto, 2004) . On the other hand, the use of common concepts and 
metaphors, such as "natural enemy" (musuh alami) or "farmers' friends" (teman 
petani), were more easily understood and facilitated knowledge formation and 
adoption (Dolly, 2009; Winarto, 2004). 

Moreover, farmers might not be sufficiently fluent in the national language to be 
able to comprehend the FFS content (DANIDA, 2011; Najjar, 2009), and in one case 
farmers' understanding was improved when facilitators used the local language (Van 
de Fliert, 1993). 
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Figure 29 Barriers to and enablers of knowledge acquisition, adoption and improved final outcomes 
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- Training delivered in a top-down 
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participants and tradition of collective 
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External/ contextual factors 


A range of contextual factors external to the programme can act to promote or 
hinder FFS effectiveness. Several of the included studies contained findings related 
to the policy context and institutional set-up, the community context and the 
infrastructure and physical context within which FFS were implemented. 

Policy context and institutional set-up 

FFS are implemented within the context of existing and emerging policies, 
institutional structures and industry activities. A range of actors are involved in 
agricultural extension, including international donors, national, regional and local 
governments, research institutions, NGOs and various industry actors. Several 
studies noted that diverging institutional incentives and objectives of these various 
actors influence how FFS are implemented in practice, with a tendency for some of 
the partidpatory elements to be eroded (Isubikalu, 2007; Pedersen et al., 2008; 
Simpson, 1997). 

There is some evidence suggesting that the existence of conflicting agricultural 
policies can act as a barrier to adoption of FFS practices. In some cases, the 
involvement of a multitude of institutions each promoting their own mandate 
resulted in each impacting on the design, content and implementation of FFS, 
crowding out the benefidary needs and interests (Isubikalu 2007, Simpson, 1997) . 
In other cases, the institutional legacy of existing extension systems influenced the 
implementation of FFS (Isubikalu 2007; Mandni et al., 2007; Van de Fliert, 1993; 
Winarto, 2004). Institutional structures assodated with earlier top-down 
agricultural extension systems, such as subsidised input schemes, trickle-down 
messages and off-the-shelf technology promotion, remained and contradided the 
bottom- up, partidpatoiy approach of FFS. 

Other evidence suggested that the power of the pestidde industry and thdr 
continued links with the extension system can act as a barrier to adoption and 
diffusion of FFS practices. In at least two cases, studies found that the pestidde 
industry maintained close links with the extension system at the local level (Mandni 
et al., 2007; Van de Fliert, 1993), paying commission to extension workers and local 
cooperatives to promote pestiddes, hampering farmers' efforts to practice I PM. 

Community context and sodal capital 

Several of the included studies highlighted the role of the community in influendng 
farmers' practices (Dolly, 2009; Machacha, 2008; Najjar, 2009; Palis, 2002; 
Simpson, 1997; Van DerWiele, 2004; Winarto, 2004). One of the hypothesised 
assumptions in the programme theory was that for sustained adoption of FFS 
practices it was important "fanners are convinced others will do the same". There 
was limited evidence to support this assumption, which was based on the suggestion 
by Feder et al. (2004a) that lack of adoption of IPM practices by neighbouring 
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fanners might curtail the effectiveness of the intervention, as pests from their fields 
may re- infest the fields of adopters, eventually leading to disadoption of I PM by FFS 
participants. In one case farmers in Indonesia found it difficult to practise new 
strategies when the rest of the community continued applying old practices 
(Winarto, 2004) . However, the reason for this seems to be that they faced 
discouragement and disbelief from their untrained peers, rather than a fear of re- 
infestation. 

In our original programme theory we suggested that a "high degree of social 
cohesion" might be an assumption that needs to hold for knowledge to diffuse to 
neighbouring farmers not participating in FFS, but we did not include it as a factor 
that might affect adoption among FFS participants. Several of the included studies 
suggested that existing social capital and reach of social networks may also influence 
adoption among FFS participants (Dolly, 2009; Gottret & Cordoba, 2004; Palis, 
2002; Simpson, 1997). In some cases high levels of social cohesion and a tradition of 
collective action from existing farmer groups may have encouraged a willingness to 
learn and succeed with the training, facilitating adoption (Dolly, 2009; Gottret & 
Cordoba, 2004; Palis, 2002) . On the other hand, in one case low levels of social 
capital and little sense of community in the FFS villages may have been a barrier to 
FFS effectiveness. 

Access to inputs 

One of the assumptions included in the programme theory was that fanners have 
access to any complementary inputs necessary to adopt FFS practices. Five studies 
highlighted issues related to availability of inputs, labour and markets (DAM DA, 
2011; Machacha, 2008; Mandni et al., 2007; Najjar, 2009; Van Der Wiele, 2004). In 
all cases the lack of access to inputs, capital and/ or markets were suggested as 
factors influencing uptake of FFS practices or final outcomes of the FFS training. 
These issues are common challenges for farmers, and are not particular to FFS 
programmes, but in some cases they appear to have remained a barrier for FFS 
farmers to adopt FFS practices and fully benefit from adoption. 

Other themes 

Sustainability 

FFS aim to provide farmers with skills which enable them to solve problems for 
themselves. An implicit assumption behind this is that the FFS training will enable 
farmers to continue with FFS practices and deal with any new challenges after they 
graduate. The sustainability of these skills and practices, along with their successful 
diffusion to broader farmer communities are an important outcome for cost- 
effectiveness of FFS interventions. Eleven out of 20 studies discussed factors 
affecting sustainability of FFS (DAM DA, 2011; David, 2007; Dolly, 2009; Friis- 
Hansen, 2008; Gottret & Cordoba, 2004; Karanja-Lumumbaetal., 2007; 
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Machacha, 2008; Simpson, 1997; Van de Fliert, 1993; Van DerWiele, 2004; 
Winarto, 2004). 


Based on the descriptive themes, all studies highlighted that ongoing support and 
follow-up are important for sustainability of FFS practices and the establishment 
and sustainability of FFS- related activities (DANIDA, 2011; David, 2007; Dolly, 
2009; Gottret& Cordoba, 2004; Karanja-Lumumbaetal., 2007; Machacha, 2008; 
Simpson, 1997; Van DerWiele, 2004; Winarto, 2004). In particular, included 
studies identified a lack of technical assistance and backstopping from researchers 
and extensionists to support farmers in continuing development of local practices 
(Dolly, 2009; Gottret & Cordoba, 2004; Simpson, 1997; Winarto, 2004). The studies 
that reported on follow-up group activities taking place suggested that active follow- 
up and continued support by the implementing agency, encouragement to establish 
farmer dubs and additional sessions on dub formations may fadlitate the 
establishment of sustainable and effective groups and practices. 

In addition, four studies also suggested that particular group characteristics may 
affect sustainability of farmer groups and FFS- related practices (DANIDA, 2011; 
David, 2007; Machacha, 2008; Van DerWiele, 2004). Studies reporting successful 
follow-up farmer activities suggested consistent membership, good leadership, 
collective goals and a supportive group environment might be important in 
maintaining FFS groups and providing impetus for further farmer- led initiatives. 

Overall, the studies suggested that lack of ongoing support and follow-up is an 
important barrier to the sustainability of the FFS approach. In the absence of formal 
follow-up, the implidt assumption is that the FFS training is suffident to enable 
farmers to continue with FFS practices and deal with any new challenges. However, 
the evidence suggested that this might not hold in most cases, as farmers expressed 
the need for additional follow-up support and technical backstopping to be able to 
continue the development of local practice. A lack of formal follow-up activities 
therefore appears to be a barrier to sustainability of FFS practices. 

An assumption missing from the original theory of change is that the group 
characteristics may play a crudal role in enabling sustainability of FFS practices and 
related activities. Based on the available evidence, it appears that choosing groups 
with common goals, good leadership and high attendance rates might be important 
enablers of sustainability of FFS- related practices. 

Perceived outcomes: gender and empowerment 

Proponents of FFS suggest empowerment is a key outcome of FFS. While few impact 
evaluation studies looked at these outcomes, empowerment and improved gender 
relations were frequently highlighted in the qualitative literature (DANIDA, 20 11; 
Dolly, 2009; Friis- Hansen, 2008; Friis- Hansen etal., 2012; Hofisi, 2003; Machacha, 
2008; Mandni et al., 2007; Najjar, 2009; Simpson, 1997, Van DerWiele, 2004; 
Winarto, 2004). Evidence based on participants' perceptions suggests FFS may 
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influence empowerment positively. Some studies also suggest participation in FFS 
may lead to women's empowerment and improve gender relations. 

IPM diffusion to neighbour farmers 

The evidence from quantitative impact evaluations overwhelmingly supports the 
notion that there is little, if any, diffusion of IPM knowledge and adoption to 
neighbouring farmers who do not participate in FFS. Over half of the studies 
included in the qualitative synthesis covered themes relevant to the diffusion of FFS 
knowledge and practices to non- participants (David, 2007; Gottret & Cordoba, 

2004; Hiller etal., 2009; Karanja-Lumumbaetal., 2007; Machacha, 2008; Mandni 
etal., 2007; Palis, 2002; Rola&Baril, 1997; Simpson, 1997; Van de Fliert, 1993; 
Winarto, 2004) . Overall they suggested both characteristics of FFS and contextual 
factors influenced diffusion, summarised in the updated programme theory (Figure 
30). 

Several characteristics of FFS may explain why practices promoted in I PM- FFS do 
not appear to diffuse spontaneously to farmers who have not participated in 
training. Four studies highlighted the complexity and the experiential nature of FFS 
learning as a barrier to diffusion (David, 2007; Mandni et al., 2007; Van de Fliert, 
1993; Winarto, 2004) . They noted that, despite high awareness of IPM by non- 
partidpants (Van de Fliert, 1993; Winarto, 2004), the skills and practices are 
complex and their experiential nature makes them difficult to convey via verbal 
communication (Mandni et al., 2007; Van de Fliert, 1993; Winarto, 2004). In two 
studies where some diffusion was observed, the findings suggested concrete 
practices were more likely to diffuse than theoretical concepts and prindples (David, 
2007; Hiller et al., 2009), with relatively simple practices, such as pruning and 
weeding techniques, being more easily disseminated. 

J ust as observability of IPM practices and their relative advantage compared with 
conventional methods was found to be important in encouraging FFS farmers to 
adopt new practices, the evidence suggested observability is important for 
convindngnon-FFS farmers to adopt IPM practices (David, 2007; Gottret & 
Cordoba, 2004; Palis, 2002; Machacha, 2008; Simpson, 1997; Winarto, 2004; Van 
de Fliert, 1993) . In theory this should be done on the plot so that non- participant 
farmers can see what is done and trained farmers may not have the time or skills to 
do so. In two cases non- FFS farmers perceived FFS practices as having relative 
advantage compared with existing practices, facilitating interest in IPM (Gottret & 
Cordoba, 2004; Hiller etal., 2009). 

For many observers the lack of diffusion of IPM practices may not be surprising. FFS 
differ from agricultural extensions that focus on disseminating knowledge of more 
simple practices, such as application of fertiliser and pesticides, or adoption of new, 
improved seeds. Instead of heavy reliance on external inputs, the practices promoted 
in FFS typically rely on analysis and use of what is available in farmers' fields and 
local ecosystems. Given the skills- based nature of the practices promoted in FFS, the 
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rate and nature of diffusion will differ from the diffusion of more simple 
technological innovations. 

One of the assumptions included in our initial programme theory was that a "high 
degree of social cohesion" would be important for diffusion. Evidence from five 
studies suggests existing levels of social capital may influence diffusion of IPM 
knowledge and practices (David, 2007; Gottret & Cordoba, 2004; Palis, 2002; Rola 
&Baril, 1997; Simpson, 1997). In some cases low levels of social capital and cohesion 
limited communication within the community and presented a barrier to diffusion 
of FFS messages (David, 2007; Simpson, 1997). On the other hand, in the 
Philippines, high levels of social capital, in particular among farmers with kinship 
ties, facilitated sharing of IPM concepts with farmers who did not participate in FFS 
training. Awareness of the main sources of social capital in a particular context, as 
well as analysis of social networks, might provide an opportunity to facilitate greater 
diffusion. 

In our original programme theory one of the assumptions for IPM knowledge and 
practices to diffuse was that formal community- building activities and training of 
FFS alumni to train other farmers were implemented. Overall the themes emerging 
from the synthesis of issues related to diffusion seemed to confirm this, and 
supported the assumption that, in the absence of formal mechanisms promoting 
diffusion, targeting "appropriate" farmers for FFS participation is particularly 
important. 

This issue was highlighted in a study from Indonesia, which found that 
socioeconomic differences between FFS participants and non- participants impeded 
diffusion (Van de Fliert, 1993). The non- representative composition of the FFS 
groups impeded interaction between participants and non- participants. FFS 
participants communicated to a "selective audience in the villages" and made no 
deliberate efforts to train other members of the community in IPM principles. 
However, another study from Indonesia found that a few inguisitive farmers played 
a prominent role in the ongoing process of knowledge formulation and transmission. 
These farmers progressively established their position within the community as 
"experts", "farmer professors" and "consultants" (Winarto, 2004, p. 351), suggesting 
some spontaneous diffusion may be possible, but that careful targeting of farmers 
with the appropriate characteristics may be necessary. 
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Figure 30_Bamers to and enablers _of_ I PM diffusion to non- FFS neighlxmr farmers and siMainability 
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6 Integrated Synthesis 


What are the effects of fanner field schools on farmers' wellbeing in terms of 
intermediate and final outcomes? What explains differences in effects across 
different contexts? Are these effects sustainable? This chapter integrates the two 
syntheses with the aim of answering these questions. 

The findings and conclusions regarding the effect of FFS on intermediate and final 
outcomes are based on the meta- analysis of quantitative impact evaluations 
addressing review question ( 1) : What are the effects of farmer field schools on 
intermediate and final outcomes for FFS participants and non- participating 
neighbours? The findings and conclusions regarding the possible reasons for 
differences in FFS effectiveness are based on the qualitative synthesis addressing 
review question (2) : What are the enablers of and barriers to FFS effectiveness, 
diffusion and sustainability? 

We updated the theory of change based on the findings from the syntheses, and also 
attempted to explain heterogeneity in findings using causal chain analysis (White, 
2009) . Figure 31 shows the updated programme theory, indicating where the causal 
chain breaks down, together with assumptions based on the identified barriers and 
enablers. 
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Figure 31 Farmer field schools theory of change: integrated synthesis of evidence 
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6.1 INTERMEDIATE OUTCOMES 


Knowledge 

No studies with a low risk of bias were identified and only three quasi- experimental 
studies were assessed as of being of medium risk of bias in measuring knowledge 
outcomes. The meta- analysis findings indicated that field school participants 
improved their knowledge of farming technology, both on average across 
evaluations as well as in all sufficiently powered individual studies. The 
improvements in scores were measured across a range of knowledge tests and also 
appeared to be both for simpler and more complex types of knowledge among the 
few studies, which differentiated these. However, the policy- relevant findings of 
effects are based on only three studies and owing to the range of methods used 
hence the heterogeneity in effects was particularly high for these outcomes. 
Knowledge outcomes improved across all FFS curricula, although they were greatest 
for IPM-FFS graduates. The finding across three medium- risk-of- bias studies 
suggested consistent increases in knowledge for FFS farmers of 0.21 standard 
deviations in test scores over comparison farmers (SMD =0.21, 95% C3 =0.07, 0.35; 
Q=5, Tau-sq=0.008, 1-sq=55%; evidence from 3 studies) (Figure 14). The 
quantitative studies did not, however, attempt to assess other aspects of capacity 
such as fanners' problem-solving capabilities, which is also an important weakness 
of the evidence on empowerment. 

While the evidence indicated knowledge acquisition among participants, it also 
suggested participant targeting and participation, facilitator training and the 
programme implementation as important barriers and enablers which might 
influence it. 

Farmer targeting and participation 

A key assumption in the FFS programme is that the "right" farmers are targeted and 
reached by the FFS. The targeted farmers must be willing and able to participate in 
FFS training throughout the full season, and be able to implement FFS practices in 
their fields. 

Overall, the review identified several key barriers related to targeting, such as 
inappropriate selection criteria and targeting procedures, and structural barriers to 
participation such as gender, poverty and cultural norms. If there is not a considered 
approach to targeting, this may result in farmers participating for the wrong reasons 
and ultimately dropping out. Alternatively, beneficiaries may not have the right 
characteristics, such as education levels, or access to land and resources, to be able 
to benefit fully from the FFS training. We were able to assess whether there were 
differences in knowledge gains among field schools which targeted farmers based on 
education levels and land size, but due to limited data provided in most included 
studies the results, though suggestive of benefits for higher socioeconomic status, 
were not conclusive. 
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General inputs 

FFS require a range of different inputs at the outset in order to establish the 
programme and start delivering services to participants. Some studies reported a 
lack of (timely) funding and inadequate provision of resources, including inputs and 
finances, as potential reasons for implementation failures (Pedersen et al., 2008; 
Najjar, 2009; Winarto, 2004). These included insufficient and incidental provision 
of inputs, discrepancies between budgeted and received payments, payment delays 
and difficulties with logistics and dissemination. In addition, selection of 
inappropriate FFS sites and a lack of adequate follow-up and support during pest 
outbreaks were noted as constraints for IPM field schools. 

Facilitator training, performance and support 

The facilitator is an important input in FFS programmes, and adequate training of 
facilitators is a key assumption in the theory of change for the FFS training to lead to 
improved knowledge and skills among participating fanners. Six of the seven studies 
that reported on training of facilitators suggested that there were issues related to a 
lack of appropriate training, resources and ongoing support of facilitators (DANIDA, 
2011; Gottret & Cordoba, 2004; Hofisi, 2003; Pedersen et al., 2008; Rola &Baril, 
1997; Simpson, 1997). Examples of deficiencies included gaps in the curriculum 
covered in the training of facilitators and lack of focus on participatory techniques 
and facilitation skills. The studies also highlighted issues related to a lack of ongoing 
support or backstopping, and a general lack of inputs to support facilitators in their 
roles. 

One issue relates to the selection and characteristics of FFS facilitators. Studies in 
Bangladesh, Indonesia, Kenya and Zimbabwe suggested appropriate criteria for 
selecting facilitators were important for identifying candidates suitable to be 
facilitators (DANIDA, 2011; Hofisi, 2003; Machacha, 2008; Winarto, 2004). These 
studies found that rather than high levels of education, characteristics such as 
personal attitude, maturity, literacy, leadership skills and experience with farming 
may be more important for facilitators to perform their role successfully. Facilitators 
not having the right characteristics from the outset appear to have been less 
successful in performing their role as a FFS facilitator. 

A big difference between farmers and facilitators in terms of socioeconomic 
characteristics might prevent farmers from fully participating and making 
suggestions or raising concerns (Simpson, 1997). Some studies also suggested that 
the gender of the facilitator is important, in particular in more conservative contexts 
(Hofisi, 2003; Van Der Wiele, 2004). 

The characteristics of the facilitators and the training they receive in turn effect 
another theme which was highlighted by several of the included studies: the 
relationship between facilitators and participants. Three of the studies suggested an 
imbalance in the farmer- facilitator relationship as a potential barrier to farmers 
learning and adoption (Isubikalu, 2007; Najjar, 2009; Simpson, 1997), with farmers 
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not feeling comfortable enough to admit when they did not understand something, 
or to voice concerns or make suggestions. 

Adoption of practices 

No studies with a low risk of bias were identified and only 11 quasi- experimental 
studies were assessed as being of medium risk of bias in measuring pesticide use and 
adoption of practices. The majority of quantitative impact studies measured 
adoption of new agricultural practices. The more traditional farmer field schools 
delivering IPM or IPPM training usually measured adoption in terms of reduced 
amounts of pesticide application, and the meta- analysis found evidence for adoption 
among these FFS beneficiaries on average across medium- risk-of- bias studies, in 
terms of reduction in pesticide use by 23 per cent on average (RR=0.77, 95% 

Cl =0.61, 0.97; Q=40, Tau-sq=0.07, 1-sq=83%; 8 studies) (). 

Other practices, whether promoted by IPM/ IPPM field schools or those utilising 
other technology, were also adopted according to the evidence, leading to average 
improvements of 0.22 standard deviations in adoption indices (SMD=0.22, 95% 

Cl =0.06, 0.38; Q=10, Tau-sq=0.02, 1-sq=80%, 3 observations) (Figure 18). 

However, there was substantial heterogeneity in findings, the most consistent effect 
sizes from the best- quality studies being for adoption of cotton crops in Asia. 
Furthermore, several prominent studies of longer-term, scaled-up projects 
estimated null or negative impacts; that is, no significant change or lower rates of 
adoption among FFS participants than non- participants. The qualitative synthesis 
suggested several key factors may influence adoption. 

Approach to learning 

A key characteristic of the design of FFS is that they follow a participatoiy approach, 
based on theories of adult education and discovery- based learning (Pontius et al., 
2002 ). 

Over half of the qualitative studies included themes pertaining to the way in which 
FFS were delivered in the field (DANIDA, 2011; Dolly, 2009; Friis- Hansen, 2008; 
Gottret &Cordoba, 2004; Hofisi, 2003; Isubikalu, 2007; Machacha, 2008; Najjar, 
2009; Pedersen et al., 2008; Rola &Baril, 1997; Van de Fliert, 1993; Winarto, 

2004) . The extent to which the FFS were delivered according to the original 
principles upon which the intervention was designed varied, ranging from 
approaches to learning similar to traditional extension, to bottom- up FFS. Where 
implementation in practice was characterised by a more top-down, transfer of 
technology approach, the qualitative evidence suggests that farmers were not able to 
see and practice IPM, which appears to have been a barrier to farmers developing 
their knowledge and analytical skills. However, we were not able to test whether this 
made a difference to FFS effectiveness in quantitative analysis due to inadequate 
reporting of intervention delivery in those (and linked) studies. 
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Related to this, the qualitative evidence suggests that due to the complexity of the 
curriculum, observability is important for farmers to trust the messages promoted in 
FFS and to develop analytical skills (Dolly, 2009; Hofisi, 2oo3; Machacha, 2008; 
Palis, 2002; Simpson, 1997; VanDerWiele, 2004; Winarto, 2004). The 
observability of IPM practices and their relative advantage compared with 
conventional methods was found to be important in enhancing farmers' confidence 
and trust in IPM messages. Similarly, in a few cases facilitators were unable to 
demonstrate observable benefits from FFS practices which may have acted as a 
barrier to adoption. 

FFS content and coverage 

For farmers to learn and adopt the practices promoted in FFS, the content of the 
curriculum should in theory be relevant to the needs of farmers, appropriate for the 
local context in terms of the crops it covers and feasible for farmers to implement in 
their fields. Most of the qualitative studies reported themes relating to the FFS 
curriculum (DANIDA, 2011; David, 2007; Dolly, 2009; Gottret & Cordoba, 2004; 
Hiller etal., 2009; Hofisi, 2003; Isubikalu, 2007; Mandni etal., 2007; Najjar, 2009; 
Palis, 2002; Simpson, 1997; Van de Fliert, 1993; Van Der Wiele, 2004). It appears 
that in most cases the FFS curriculum was indeed relevant and appropriate to the 
local context (DANIDA, 2011; Dolly, 2009; Gottret & Cordoba, 2004; Hiller et al., 
2009; Hofisi, 2003; Mandni et al., 2007; Van de Fliert, 1993), for instance by 
responding to local concerns over economic and environmental costs of pestiddes 
(Dolly, 2009; Mandni et al., 2007). 

However, in some cases the FFS curriculum appears to have lacked local relevance 
(Isubikalu, 2007; Najjar, 2009; Simpson, 1997), and the major barrier in these cases 
seemed to be that the curriculum was not suffidently tailored to the needs and 
resources of the farmers it was targeting. A failure to incorporate a broader range of 
concerns in the curriculum was perceived by farmers to be a weakness in several 
studies (DANIDA, 2011; Mandni et al., 2007; Najjar, 2009; Van de Fliert, 1993; 
Winarto, 2004). 

In some cases the complexity of IPM made it difficult for farmers to implement all 
the IPM practices on their crops (Gottret & Cordoba, 2004; Van de Fliert, 1993; 
Winarto, 2004) . Some of the analytical tools used were too complex and perceived 
as impractical in relation to the time, energy and resources they required (Winarto, 
2004). 

Related to the complexity of the curriculum is the discourse and language used for 
communication in FFS, which needs to be in a language farmers understand 
(DANIDA, 2011; Dolly, 2009; Najjar, 2009; Van de Fliert, 1993; Winarto, 2004). 

Policy context 

Field schools are inevitably embedded within a policy and community context which 
can ad to promote or hinder FFS effectiveness. Some evidence suggested that the 
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existence of conflicting agricultural policies can act as a barrier to adoption of FFS 
practices. In some cases, the involvement of a multitude of institutions each 
promoting their own mandate resulted in each impacting on the design, content and 
implementation of FFS, crowding out the beneficiary needs and interests (Isubikalu 
2007; Simpson, 1997). 

In other cases, the institutional legacy of existing extension systems influenced the 
implementation of FFS (Isubikalu 2007; Mandni et al., 2007; Van de Fliert, 1993; 
Winarto, 2004). Institutional structures associated with earlier top-down 
agricultural extension systems, such as subsidised input schemes, trickle- down 
messages and off-the-shelf technology promotion, remained and contradicted the 
bottom- up, participatory approach of FFS. 

There was also some evidence suggesting the power of the pesticide industry and 
continued links with the extension system acted as a barrier to adoption and 
diffusion of FFS practices. At least two studies found that the pesticide industry 
maintained dose links with the extension system at the local level (Mandni et al., 
2007; Van de Fliert, 1993), paying commission to extension workers and local 
cooperatives to promote pestiddes, hampering farmers' efforts to practise IPM. 

Several studies suggested that policy context was key for FFS effectiveness and 
sustainability, although no studies provided any systematic analysis (Praneetvatakul 
&Waibel, 2006; Pananurak, 2010; Wu, 2010). In particular, high levels of subsidies 
for inputs, inducting pestiddes, might have provided strong incentives against IPM 
(Pananurak, 2010). Incentives to use pestiddes, such as subsidies, and different 
market- based as well as institutional disincentives to the adoption of IPM, may have 
reduced adoption rates (Praneetvatakul &Waibel, 2006; Pananurak, 2010). This 
was the case in Thailand where, after a period of high-level support for FFS in the 
Department for Agricultural Extension, a change of leadership in the department 
reversed priorities towards pestidde- based crop protection and the FFS programme 
declined (Praneetvatakul &Waibel, 2006). In China, plant protection stations at 
township and county levels were found to be involved in pestidde sales due to 
shortages of operation funds, resulting in a diversion of human resources or even 
outright subversion of extension agents' attitudes towards pestidde use (Wu, 2010). 

Community and sodal capital 

Another emerging theme was the potential role of existing sodal capital in 
influendng FFS effectiveness. Several of the qualitative studies highlighted the role 
of the community in influendng farmers' practices (Dolly, 2009; Machacha, 2008; 
Najjar, 2009; Palis, 2002; Simpson, 1997; VanDerWiele, 2004; Winarto, 2004). 

There was limited evidence to support the assumption that sustained adoption of 
FFS practices requires community- wide uptake (Feder et al., 2004a). In one study, 
farmers in Indonesia found it difficult to practise new strategies when the rest of the 
community continued applying the existing practices (Winarto, 2004). However, the 
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reason for this seems to be that they faced discouragement and disbelief from their 
untrained peers, rather than a fear of re- infestation. 

Several of the qualitative studies suggested existing social capital and reach of social 
networks may also influence adoption among FFS participants (Dolly, 2009; Gottret 
& Cordoba, 2004; Palis, 2002; Simpson, 1997). In some cases high levels of social 
capital and a tradition of collective action from existing farmer groups may have 
encouraged a willingness to learn and succeed with the training, and facilitated 
adoption (Dolly, 2009; Gottret & Cordoba, 2004; Palis, 2002). On the other hand, in 
one case low levels of social capital and little sense of community in the FFS villages 
may have been a barrier. 

Access to inputs 

Five studies highlighted issues related to availability of inputs, labour and markets 
(DANIDA, 2011; Machacha, 2008; Mandni et al., 2007; Najjar, 2009, Van Der 
Wiele, 2004). In all cases the lack of access to inputs, capital and/ or markets were 
suggested as factors influencing uptake of FFS practices or final outcomes of the FFS 
training These issues are common challenges for farmers, and are not particular to 
FFS programmes, but they appear to have remained a barrier for FFS fanners to 
adopt FFS practices and benefit fully from adoption. Indeed, the analysis suggested 
field schools involving farmers who were of higher socioeconomic status (measured 
by education and landholdings) tended to have greater impacts in terms of adoption 
as well as agricultural outcomes. 


6.2 FINAL OUTCOMES 


Agriculture and other outcomes 

No studies with a low risk of bias were identified and 11 quasi- experimental studies 
were assessed as bang of medium risk of bias in measuring agricultural outcomes. 
The meta- analysis found evidence that FFS improved agricultural outcomes among 
participants, as measured by a 13 per cent increase in yields on average across 
medium-risk-of-bias studies (RR=1.13, 95% Cl =1.04, 1.22; Q=53, Tau-sq=0.008, 1- 
sq=81%; 11 observations) (Figure 20) and 19 per cent increase in profits or net 
revenues (RR=1.19, 95% Q =1.11, 1.27; Q=l, Tau-sq=0, l-sq=0%; 2 observations) 
(Figure 22) . However, the analysis also found evidence for publication bias for yields 
outcomes, suggesting effects may be smaller or non- significant. 

The estimated impact on net revenues was proportionately greater than yields, a 
likely consequence of the combination of both improved agricultural output and 
reduced costs in terms of pesticide use. The effects were similar both for I PM/ IPPM 
field schools and field schools promoting other curricula, although the extent of 
evidence for FFS promoting these other curricula was relatively weak. However, 
impacts of FFS do not appear to be significantly positive for longer follow-up periods 
(greater than two years) and scaled- up programmes implemented at national level. 
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There was no evidence that longer-term programmes implemented at national scale 
are effective, although only two such programmes in Asia have been evaluated. 

Authors have suggested that FFS are likely to result in substantial benefits only in 
areas where farmers overuse pesticides, practise intensive methods of farming or 
have so far ignored economic considerations in their decisions to apply pesticides 
(Praneetvatakul et al., 2007) . Furthermore, barriers in going to scale may arise due 
to problems in recruiting and training sufficient numbers of facilitators, as noted 
above. 

The qualitative evidence suggested ongoing support and/ or follow-up are important 
for sustainability of the FFS approach, including sufficient technical support and 
backstopping from researchers and extensionists to allow fanners to continue the 
development of local practices (DAMDA, 2011; David, 2007; Dolly, 2009; Gottret & 
Cordoba, 2004; Karanja-Lumumbaetal., 2007; Machacha, 2008; Simpson, 1997; 
Van Der Wiele, 2004; Winarto, 2004). 

The studies which reported on follow-up group activities taking place suggested that 
active follow-up and continued support by the implementing agency, 
encouragement to establish farmer dubs and additional sessions on dub formations 
may fadlitate the establishment of sustainable and effective groups and practices. 
Existence of support and follow-up was also found to contribute to the 
establishment and sustainability of FFS- related activities. 

Several factors were highlighted as important for the sustainability of FFS groups 
following graduation. These include consistent membership partidpation, 
leadership, collective goals and activities, and group support and validation 
important in building up confidence in FFS practices of FFS graduates. There was 
also a suggestion that reimbursing partidpants for FFS attendance may have 
undermined sustainability of the FFS groups. 

The evidence on outcomes relating to health and the environment was very limited. 
No studies of suffident internal validity measured health outcomes. A number of 
studies measured the environmental impact quotient score, which is an indirect 
measure based solely on estimates of pestidde use, finding improvements for FFS 
farmers only (RR=0.61, 95% Cl =0.48, 0.78; Q=3, Tau-sq=0.01, 1-sq=33%; 3 
observations) (Figure 24). 


Empowerment and gender relations 

Proponents of FFS suggest empowerment is a key outcome of FFS. Few quantitative 
studies were able to report on aspects of empowerment such as self-esteem. One 
medium-risk-of-bias study in Peru (Van Rijn et al., 2010) estimated increases in the 
probability of feeling capable of solving problems in the field, feeling comfortable in 
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giving an opinion, and participating in the community (RR=2. 13, 95% Q =1.46, 3. 12) 
(Figure 26). 

While few impact evaluation studies looked at these outcomes, empowerment and 
perceived improvements in gender relations were frequently highlighted in the 
qualitative literature (DANIDA, 2011; Dolly, 2009; Friis- Hansen, 2008; Friis- 
Hansen et al., 2012; Hofisi, 2003; Machacha, 2008; Mandni et al., 2007; Najjar, 
2009; Simpson, 1997; Van DerWiele, 2004; Winarto, 2004). Evidence from these 
studies based on participants' perceptions suggests FFS may influence feelings of 
empowerment and gender relations positively. However, there is a lack of causal 
evidence to support these findings. 


6.3 DIFFUSION TO NON- PARTICIPATING NEIGHBOUR 

FARMERS 


The quantitative impact evidence overwhelmingly supported the notion that there 
was little, if any, diffusion of integrated pest management (IPM) to non- 
participating neighbour farmers through their interactions with FFS participants or 
other outreach activities. The impact evaluations did not provide any evidence for 
diffusion of knowledge to neighbouring farmers. Only four high-risk- of- bias studies 
estimated impacts which were insignificantly different from zero, individually and 
pooled. Consequently, there was no evidence for adoption among neighbours in 
terms of pesticide use (RR=0. 95, 95% Cl =0.64, 1.39; Q=45, Tau-sq=0.14, 1-sq=91%; 
5 observations) on average across studies (no studies measured adoption of other 
practices for neighbours), nor any significant changes in outcomes for neighbour 
farmers in terms of yields (RR=1.02, 95% Cl =0.97, 1.08; Q=9, Tau-sq=0.002, 1- 
sq=66%; 4 observations) (Figure 32) . 

Some evidence suggested that there may have been diffusion of simple messages in 
the short term, where FFS participants were relatively well educated, but not in the 
long term and no studies collected data for more complex practices. 

Over half of the studies included in the qualitative synthesis covered themes relevant 
to the diffusion of FFS knowledge and practices to non- participants (David, 2007; 
Gotret & Cordoba, 2004; Hiller et al., 2009; Karanj a- Lumumba et al., 2007; 
Machacha, 2008; Mandni etal., 2007; Palis, 2002; Rola&Baril, 1997; Simpson, 
1997; Van de Fliert, 1993; Winarto, 2004). Overall they suggested both 
characteristics of FFS and contextual factors act as barriers to diffusion. 
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Figure 32 Summary meta- analysis findings for IPM-FFS neighbours 
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> 1.02(0.97,1.08) 

Revenues 

2 studies (l-squared = 0.0%, p = 0.424) 

0 1.09(1.03,1.15) 
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2 studies (l-squared = 0.0%, p = 0.462) < 

1 

1 - 44 (°- 88 > 2 - 35 ) 

1 1 1 


.5 1 2 3 6 

Favours comparison Favours neighbour farmers 


Notes: Response ratio (RR) measures ratio of outcomes among FFS neighbours to comparison group. * Variable 
inverted. 


Characteristics of FFS: Complexity, observability and relative 
advantage 

Several characteristics of FFS may explain why practices promoted in FFS did not 
spontaneously diffuse in the community. Studies in Cameroon, India and Indonesia 
highlighted the complexity and the experiential nature of FFS learning as a barrier 
to diffusion (David, 2007; Mandni et al., 2007; Van de Fliert, 1993, Winarto, 2004). 
They noted that, despite high awareness of I PM by non- participants (Van de Fliert, 
1993; Winarto, 2004), the skills and practices are complex and their experiential 
nature makes them difficult to convey via verbal communication (Mandni et al., 
2007; Van de Fliert, 1993; Winarto, 2004). In two studies in Cameroon and Kenya 
where some diffusion was observed, the findings suggested concrete practices were 
more likely to diffuse than theoretical concepts and prindples (David, 2007; Hiller 
et al., 2009). The studies also found that relatively simple practices, such as pruning 
and weeding techniques, were more easily disseminated than more complex 
practices, as also found in one quantitative study in Bangladesh (Ricker- Gilbert et 
al., 2008). 


J ust as observability of IPM practices and their relative advantage compared with 
conventional methods were found to be important in encouraging FFS farmers to 


152 


adopt new practices, the evidence suggested observability of FFS practices were 
important for convincing non- FFS fanners to adopt FFS practices (David, 2007; 
Gottret & Cordoba, 2004; Palis, 2002; Machacha, 2008; Simpson, 1997; Vande 
Fliert, 1993; Winarto, 2004). 

Contextual factors 

Five qualitative studies suggested existing levels of social capital influenced diffusion 
(David, 2007; Gottret & Cordoba, 2004; Palis, 2002; Rola&Baril, 1997; Simpson, 
1997). 

In some cases low levels of social capital and cohesion limited communication 
within the community and presented a barrier to diffusion of FFS messages (David, 
2007; Simpson, 1997). On the other hand, in the Philippines, high levels of social 
capital, in particular among farmers with kinship ties, facilitated sharing of IPM 
concepts with farmers who did not participate in FFS training. Awareness of the 
main sources of social capital in a particular context, as well as analysis of social 
networks, might provide an opportunity to facilitate greater diffusion. 

Targeting FFS farmers 

Meta- analysis suggested that, while IPM messages did not spill over to neighbour 
farmers on average, there may have been diffusion in terms of reduced pesticide use 
and improved yields among cotton growers in the short term (less than two years 
after FFS training), where field schools targeted participants who were more highly 
educated. The findings should be interpreted with caution due to the limited sample 
size. However, those studies that examined sustainability found that any initial 
adoption among neighbouring farmers, in terms of knowledge gains, pesticide use 
and yields, fell considerably over time. 


6.4 AUTHORS' CONCLUSIONS 


In this review we have synthesised evidence on the effectiveness of farmer field 
schools in improving intermediate outcomes (farmers' knowledge, adoption), and 
final outcomes (yields, revenue, health, empowerment, environmental outcomes). 
Farmer field schools are complex interventions implemented using different 
methods of delivery in a range of different contexts. For the review to be more useful 
for decision-makers, we also synthesised qualitative evidence on barriers to and 
enablers of the effectiveness of FFS programmes. 

The results of meta- analysis provide evidence that FFS are effective in improving 
intermediate and final outcomes among farmers participating in the training. 
Although no single study reported on all outcomes, this finding is consistent across 
the theory of change - farmers appear to learn as a result of training, they adopt 
simple practices such as reduced pesticide use in the case of IPM, and they 
experience improvements in outcomes such as yields and revenues. The finding also 
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appears to hold for different types of FFS curricula including the more standard IPM 
as well as I PPM and fanning approaches based on other curricula. 

The vast maj ority of evaluations were assessed as being of high risk of bias due to the 
approach to counterfactual generation. No studies randomly assigned farmer groups 
or villages to the programme, an approach which is very feasible for FFS, and most 
did not use rigorous quasi- experimental designs and methodologies. Due to the 
nature of the evidence, it is not possible to say at this point whether the benefits 
were sustained, as few studies have collected follow-up data for more than two years. 
And the evidence suggests that the few FFS programmes which have been 
implemented on a national scale have not been effective in improving farmer 
outcomes. Data from more rigorous evaluations with longer follow-ups are urgently 
needed. 

Evidence suggests that there are no significant effects of diffusion of IPM to 
neighbouring farmers who did not participate in the FFS training. For many 
observers the lack of evidence for sustained diffusion of FFS practices in the case of 
IPM may not be surprising FFS differ from other interventions such as those that 
focus on disseminating knowledge of more simple practices, for instance application 
of fertiliser and pesticides, or adoption of new, improved seeds. The qualitative 
evidence suggests that both characteristics of FFS and contextual factors are the 
primary barriers to diffusion. In particular, the complexity of FFS practices, the need 
for experiential training, and the importance of observing FFS practices to recognise 
their relative advantage over conventional farmer practices, appear to act as barriers 
to spontaneous diffusion. Existing levels of social capital and the reach of social 
networks are also potentially important. 
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7 Implications 


7.1 IMPLICATIONS FOR POLICY 


The systematic review provides evidence that fanner field schools can be effective in 
the short term in improving fanner knowledge and adoption of better practices, and 
improving yields, revenues and environmental outcomes among field school 
beneficiaries. These beneficial impacts have been recorded across different types of 
field school, including those incorporating IPM, IPPM and other techniques. 

The impacts on agricultural outcomes are potentially of substantive importance to 
farmers, in the region of a 13 per cent increase in yields and a 20 per cent increase in 
profits (net revenues) . The effects on revenues appear to be particularly large when 
FFS are implemented alongside complementary upstream or downstream 
interventions (access to seeds and other inputs, assistance in marketing produce). 

For IPM field schools there is no convincing evidence for diffusion to neighbour 
farmers who live in the same communities as field school graduates. If diffusion is 
important for sustained adoption, this is a potentially important weakness of the 
approach as it has been implemented thus far. 

There is also no evidence that the impacts on trained fanners are sustained over 
time or scalable. Studies of scaled- up programmes measuring outcomes over the 
longer term (more than two years post- training) do not find any evidence of effects. 

As a method of rural adult education, the farmer field school appears suited for 
gradual scale-up with a dear focus on ensuring local institutionalisation (i.e. 
favouring depth of coverage in each community over geographical breadth) . On the 
other hand, FFS seem unsuited to solve the problems of large-scale extension from 
the past. The highly intensive nature of the training programme, the relative 
successes in targeting more educated fanners as compared with disadvantaged 
groups, and failures in promoting diffusion of IPM practices, suggest that the 
approach is not cost-effective compared with agricultural extension in many 
contexts, except where existing fanning practices are particularly damaging. 


Stronger polides and regulatory measures may be necessary to counteract the 
activities of the pestidde industry, including the promotion and sale of pestiddes by 
extension workers. New polides fadhtatingpartidpatory agricultural extension 
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approaches, replacing earlier extension policies aimed at promoting off-the-shelf 
technologies and input packages, may also be necessary. 


7.2 IMPLICATIONS FOR PROGRAMME IMPLEMENTATION 


The review also highlighted factors which are relevant to implementation of farmer 
field school projects and programmes. 

Training of facilitators: In terms of intervention delivery, the training and 
performance of FFS facilitators is important for the success of FFS and this does 
require considerable initial investment. This is likely to be particularly problematic 
for programmes implemented at scale. Recruitment of facilitators should take into 
account personal attitude, maturity, literacy, leadership skills, knowledge in local 
language and experience with fanning. In many contexts the gender of the facilitator 
should be carefully considered. Training of facilitators should provide sufficient 
substantive expertise in IPM or other relevant practices as appropriate to the local 
context. The training should also focus on participatory techniques and facilitation 
skills and emphasise the need to use language and concepts with which farmers are 
familiar. Facilitators should have access to ongoing support and backstopping from 
supervisors and technical experts connected to local research centres. 

Field school design and approach: Our underlying theory of change suggests FFS 
should be delivered according to a participatory and discovery- based approach to 
learning, including opportunities for farmers to experiment and observe new 
practices, where the main objectives are skills development and other forms of 
empowerment. Regular monitoring of facilitators may help to ensure this happens in 
practice, and to identify schools where additional support is required. 

Consultation and needs assessments are important to ensure the appropriateness of 
the curriculum to the local context, as well as its relevance to farmers' needs. The 
curriculum and crops covered in FFS should also be adapted according to the local 
agricultural system and the needs of the farmers targeted by the programme. 
Curricula need to deal with the major challenges facing farmers. In most cases these 
challenges will be multifaceted and there is a need to balance comprehensiveness 
with the ability to cover all issues in sufficient depth. A cumulative approach over 
several seasons might be preferable. 

Complementary interventions, such as access to finance and inputs such as 
improved seeds, as well as assistance with marketing, may improve profits (net 
revenues). 


Efficient monitoring and evaluation systems should be put in place alongside FFS 
implementation to ensure adequate and timely delivery of resources, and to ensure 
that sites selected for FFS are appropriate. 
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Dissemination of information to fanners about the resources allocated to FFS might 
enhance accountability and improve the likelihood of resources reaching FFS. 

Targeting: Proponents of FFS have recommended targeting younger farmers, those 
with greater land endowments, and women, favouring those with relatively low 
opportunity costs of labour and/ or farmers with relatively high pesticide costs. 
Indeed, Davis et al. (2012) suggest that the FFS approach could serve as a key 
strategy to provide agricultural extension services to female farmers in Africa, whose 
access to agricultural extension is generally poor. Women are frequently targeted by 
FFS, but field schools will be less effective if women are from households where they 
are not in a decision-making position (as was found in several of the studies 
included in this review) . There were also some problems highlighted with targeting 
youth that cannot dedicate their time to the FFS plot or their fields. 

Analysis suggests field schools involving relatively better- educated farmers tend to 
produce larger effects on adoption and agricultural outcomes. Nevertheless, FFS 
may be more effective if they adopt a considered approach to targeting based on who 
the intended beneficiaries are and in-depth analysis of the local context, including 
agricultural practices, cultural norms and gender relations. Strategies which include 
sensitisation around an interest to learn new skills might be successful at recruiting 
relevant participants. 

If the aim is to include women and disadvantaged members of the community, the 
implementing agency may need to tailor the intervention to enable their 
participation in the programme. The curriculum needs to be relevant and consistent 
with the needs and opportunities of women and the poor. For example, in contexts 
where women are primarily responsible for growing subsistence crops, a curriculum 
that covers only commercial crops is unlikely to attract women participants. 

Sensitisation exercises in the community might facilitate participation of 
disadvantaged groups, for example where men do not allow their wives to 
participate in the training because they do not see the benefits or are uneasy about 
the prospect of their wives working with other men. 

Institutional actors involved in FFS should consider beneficiary needs and interests 
in the design and implementation of the FFS programme. 

Sustainability: Formal support and encouragement of FFS alumni, including 
technical assistance and backstopping may be important for the sustainability of 
FFS practices and related activities. Working with FFS groups to support common 
goals, good leadership and high attendance rates might facilitate sustainability of 
FFS activities after the end of the training. Particular contexts may be more relevant 
for sustainability such as areas with dear overuse of pestiddes and therefore dearer 
benefits from IPM adoption. 
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Adoption of IPM by neighbouring farmers: Awareness of the main sources of social 
capital in a particular context and analysis of social networks to inform targeting 
may enhance the efficiency of farmer- to- farmer diffusion of FFS practices. 
Complementary interventions such as mass media campaigns are likely to improve 
diffusion for simple messages (such as "no early spray") only. However, given the 
skills- based nature of the practices promoted in FFS, formal community- building 
activities, support and attempts to institutionalise the approach, to encourage FFS 
graduates to train other farmers, are likely to be needed for any diffusion to 
neighbours. However, the evidence does not suggest that such approaches have been 
effective so far. 


7.3 IMPLICATIONS FOR RESEARCH 


A large number of impact evaluations aim to demonstrate the effects of farmer field 
schools, mainly on agriculture outcomes. A smaller number examine other outcomes 
such as health, empowerment and environmental impacts. However, the majority of 
available studies are not sufficiently rigorous to inform policy. 

Most quantitative impact evaluations were classified as having a high risk of bias. In 
many studies, no serious attempts were made to control for confounding through 
statistical matching or in other statistical analysis, and in many cases statistical 
significance tests were not reported. Many studies stated that they matched 
communities although this matching does not appear to be statistically rigorous. The 
consequence was the systematic overestimation of impacts, as demonstrated in the 
analysis for nearly all outcomes (Figure 33) . We were not able to identify any high- 
quality impact evaluations which we could classify as of low risk of bias in causal 
attribution. No studies used randomised assignment, although cluster randomised 
trials are feasible for studying FFS impacts on participating communities. More 
studies using rigorous counterfactuals, especially those based on prospective 
assignment (randomised or otherwise), allocated at cluster level to measure 
spillovers, together with long-term follow-ups to determine sustainability, are 
needed. 

Impact evaluations should routinely report information needed to calculate effect 
sizes in particular means and standard deviations of outcome variables and sample 
sizes including numbers of clusters and cluster sample sizes per treatment arm. 
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Figure 33 High- risk-of- bias studies produce systematically larger effects 


Outcome variable (by study risk of bias) 


Number of studies included in analysis 


RR (95% Cl) 

Reduction in pesticide use* (high risk of bias) 
13 studies 

o 

1.58 (1.21,2.07) 

Reduction in pesticide use* (medium risk of bias) 
8 studies , 

O 

1.29 (1.03, 1.63) 

Yields (high risk of bias) 
19 studies 

o 

1.26 (1.15, 1.39) 

Yields (medium risk of bias) 
11 studies 

o 

1.13 (1.04, 1.22) 

Revenues (FFS only) (high risk of bias) 
8 studies 

o 

1.34 (1.17, 1.55) 

Revenues (FFS only) (medium risk of bias) 

2 studies 

1 

o 

r~ 

1.19 (1.11, 1.27) 
1 


.5 1 2 3 

Favours comparison Favours FFS farmers 


Note: * variable inverted 


A body of high-quality, theory- based impact evaluations that report and analyse a 
hypothesised causal chain is needed to improve policy relevance and usefulness of 
findings. Evaluations should collect and report data on intermediate and endpoint 
outcomes, and incorporate qualitative assessment of implementation processes 
where possible. Studies should also assess whether FFS have heterogeneous effects 
across sub-groups of farmers. 

Studies need to measure a broader range of policy- relevant outcomes. More studies 
are needed to assess the effects of attending FFS on empowerment outcomes. In 
addition to measuring empowerment quantitatively, such studies should also 
attempt to assess the mechanisms underlying any empowerment effects. In addition 
to empowerment, some FFS also aim to improve health and environmental 
outcomes, but few existing studies included in our review have measured these. 
Future studies should collect data on these outcomes using high- quality research 
methods, drawing on the programme theory and consulting with stakeholders to 
identify all relevant important outcomes. 


The analysis presented here suggests that causal chain analysis can be useful. 
However, studies need to measure consistently the full range of intermediate and 
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final outcomes along the causal chain in order to enhance understanding of where 
the theory of change breaks down in particular programmes. Better access by donors 
and implementers to project documents and evaluation reports would also enable 
more rigorous analysis of programme design and implementation. In addition, as 
shown by the large number of studies that were excluded due to a lack of detail on 
methods, qualitative evaluations need to report aspects of the research process in 
greater detail to allow users to assess their credibility and trustworthiness. In 
particular, dear reporting on objectives, on methods of sampling, data collection 
and analysis are needed. Use of structured abstracts and more structured reporting 
of the full text of primary studies will also enhance our ability to assess the 
credibility of qualitative research and to use that research in evidence syntheses. 

Few studies reported on the subjective views and experiences of FFS fadlitators. 

This is a weakness of the existing evidence base and future studies should indude 
perspectives of FFS fadlitators and, where relevant, agricultural extension workers. 


7.4 LIMITATIONS OF THE REVIEW 


Due to the large number of quantitative impad studies induded in this review, we 
did not undertake double- coding for studies assessed to be of high risk of bias, nor 
did we contact authors to obtain additional information not reported in these high- 
risk- of- bias studies. The data from these studies are particularly unreliable, hence 
we have not drawn any conclusions for policy based on those findings. The review 
reports information on the extent of agreement between coders, although tests for 
inter- rater reliability based on Cohen's kappa were not calculated. 

A second important limitation is that the meta- analysis has involved the synthesis of 
quantitative studies that come from different study designs, estimating both 
bivariate and multivariate relationships, and multivariate models with different 
study covariates. While all medium- risk-of- bias studies, which were used to make 
implications for policy, did report results from multivariate analyses, these effect 
sizes are not truly comparable due to the different covariates included in each model 
( see Appendix D). 

There are limitations to the qualitative synthesis both due to the quality of the 
included studies, as well as the way in which we conducted the review of those 
included studies. As highlighted by the quality appraisal, there are dear limitations 
to the evidence base, and therefore the findings of the synthesis should be 
interpreted with caution. 

There remains some disconnect between the effectiveness synthesis and the 
qualitative synthesis, as there is not a dear link between the interventions studied in 
the quantitative and qualitative studies. We attempted to fink the studies by FFS 
programme name, but information on implementation was insuffidently reported in 
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either the included impact and qualitative studies or the project documentation we 
were able to identify. We are not able to conclude strongly about heterogeneity in 
effects with respect to the moderating factors identified in the qualitative synthesis, 
particularly FFS design and implementation, and therefore we are not able to 
provide strong policy guidance on how to amend programme design or 
implementation to maximise impacts. There are no cases where we have 
information about intervention effects based on rigorous counterfactual analysis and 
detailed information about implementation and other barriers and enablers that 
may moderate intervention effectiveness. 


7.5 DEVIATIONS FROM PROTOCOL 


The protocol suggested we would include all qualitative and quantitative studies 
relevant to assessing the barriers to and enablers of FFS effectiveness to answer 
review question ( 2) . However, during the review process we realised that 
synthesising such a broad range of studies presented challenges for data extraction, 
quality appraisal and synthesis. Hence we revised the inclusion criteria for barriers 
and enablers synthesis to exclude correlation, regression and simulation or 
modelling studies. However, we collected additional data on farmer characteristics 
and programme targeting from all effectiveness studies (including linked papers) to 
supplement the quantitative analysis for review question ( 1), conducting moderator 
analyses across all studies where these data were available. Some of the moderator 
variables, such as on intervention components, were identified a posteriori after the 
qualitative synthesis had been completed. For mixed methods studies that included 
an impact evaluation component, we assessed, coded and synthesised only those 
sections relevant for the qualitative synthesis (i.e. those sections that presented 
evidence about the barriers to and enablers of FFS effectiveness) . The impact 
evaluation evidence in these studies was separately assessed for inclusion for the 
quantitative review component. 
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Appendix A 


GLOBAL PROJECT PORTFOLIO REVIEW 


The global status of farmer field schools 

The following analysis is intended to provide an overview of the way in which the 
FFS approach has been developed, the manner in which it has been implemented, 
the objectives that FFS projects have pursued, and the types and groups of people 
that have participated in them. The portfolio review is based upon a systematic 
search for relevant published and unpublished literature, the use of predetermined 
inclusion criteria for admissable data, and a systematic coding and data extraction 
process (for hill details of the research methodology, see Appendix Al) . Although the 
portfolio's research methodology aimed to cast as wide a net as possible, not all FFS 
project documentation was freely available and therefore the portfolio does not 
represent a comprehensive review of all global FFS projects to have been carded out 
to date. 41 

The search process employed in this portfolio review of projects is shown in Figure 
Al. As far as possible, where projects were clearly identified, any data relating to a 
project was consolidated in a single portfolio entry. Hence, there should be no 
multiple entries for the same projects. Careful checks were made to avoid double- 
counting (by comparing project names, locations and other reported details). 

By combining the information available in the portfolio from our searches 42 with 
that from Braun et al.'s (2006) dataset, 43 we estimate that, historically, somewhere 
in the range of 400,000 to 475,000 farmer field schools 44 have been undertaken 
across all low- and middle- income countries (LMICs), producing 10- 12 million 
graduating farmers. 45 The latter figure is comparable with Braun and Duveskog's 
(2008) estimate that by 2008 between 10 million and 20 million farmers had 
graduated from farmer field schools globally. The figures above should be 


41 In particular, the FAO were unfortunately unable to release internal documentation on FAO- funded 
FFS projects. The information provided by different source documents is also of a varying standard and 
completeness. 

42 See Appendix Al, Data extraction and analysis ( 1) . 

43 The authors would like to thank Amaud Braun for sharing this data with us. 

44 A school is typically a single cohort receiving FFS training for a determined time. Typically this would 
be in one (or occasionally) multiple specified locations (though not wide geographic areas). The total 
figures are approximate given problems in identifying the unit of analysis, and acknowledged as such in 
the text. A school was defined in one of two ways: some projects reported that they included a given 
number of schools; others reported a number of participants only and this figure was divided by 25 to 
produce a proxy number of schools. 

45 See Appendix Al, Data extraction and analysis (2) . 
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interpreted as lower bound estimates, given that the portfolio and the Braun et al. 
(2006) dataset still only capture a proportion of all historic FFS initiatives. 


Figure Al FFS global portfolio research overview 
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(2006)* review of FFS programmes 
was integrated, producing a total 
of 337 portfolio entries. 


Combining analysis from the portfolio with data again taken from the Braun et al. 
(2006) FFS review indicates that farmer field schools have been introduced to at 
least 90 LMICs worldwide, as illustrated in Figure A2. 


Figure A2 Coverage of farmer field schools in LMICs 46 



■ 1995-1999 

■ 2000-2004 

□ 2005-2009 

□ 2010 onwards 

□ Start date WA 


46 The map in Figure A2 was created using using the chartsbin website: www.chaitsbin.com 
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Farmer field schools were first introduced in Indonesia in 1989. With the support of 
the FAO, they were then implemented in various other South-East Asian countries. 
The popularity of the FFS approach subsequently led to its adoption and adaptation 
in countries around the world, especially in Africa. Over half of all FFS projects in 
the portfolio were based in Africa, although the maj ority of beneficiaries ( around 
60%) were Asian. This illustrates both the extent to which the FFS approach has 
been implemented in Africa and the comparatively larger scale of Asian- based FFS 
programmes. Figure A3 illustrates the growth in implementation of the FFS 
approach since the early 1990s, with a marked growth in the number of projects in 
Asia and especially in Africa during the 2000s and beyond. 

The average duration 47 of the projects in the portfolio was around four and a half 
years, although the most common proj ect duration was three years. The actual FFS 
duration for most of the benefiting farmers was most commonly a single season, 
although there were a significant number of proj ects for which training lasted a year 
or longer. Thus, a typical project from the portfolio would last three years and would 
contain several FFS, each lasting a single season. 


Figure A3 Cumulative number of farmer field school projects implemented 
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Note: The trend calculated here is based on project start dates and includes data for the 99% 
of portfolio projects for which a start date could be established. 

Funding, implementation and costs 

Agricultural extension activities provide an opportunity to reach some of the poorest 
in the world and provide them with the knowledge and tools to increase productivity 
and improve livelihoods. Organisations with a special interest such as the FAO or 
IFAD have led the way, but host governments and other multilaterals, bilaterals and 
NGOs have also enthusiastically provided funding for a range of FFS proj ects around 
the world ( Figure A4) . 


47 A distinction is made here between project duration, or the number of months or years that an FFS 
proj ect lasts for, and the FFS duration, or the length of time for which a given cohort of benefiting 
farmers receive training. 
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Figure A4 Funding organisations 



Notes: * Such as UN bodies and multilateral banks, t Induding agencies such as DFID or 
AusAID or direct funding from foreign governments. $ Induding producer groups, farmer 
collectives and assodations. Projects may receive funding from multiple organisations, hence 
total percentage sums to greater than 100 per cent. 

Figure A5 provides a breakdown of the proportion of projects in which different 
types of organisation have helped to coordinate or implement projects on the 
ground. Host governments have been involved in the implementation of over half 
the projects in the portfolio, while NGOs and multilateral agencies have also played 
a big part. Research institutes with a specific interest in FFS have also played a 
significant role in the implementation and/ or coordination of projects. For example, 
the International Potato Center (QP) played a key role in the implementation of a 
number of FFS projects in Peru which have specialised in potato farming (Zuger, 
2004). 


Figure A5 Coordinating and implementing organisations 



Notes: *Such as UN bodies and multilateral banks, t Induding agendes such as DFID or 
AusAID or direct funding from foreign governments. $ Induding producer groups, farmer 
collectives and assodations. Projects can be implemented by multiple organisations, hence 
total percentage sums to greater than 100 per cent. 
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Van den Berg and J iggins (2007) report costs of FFS projects per graduating farmer 
ranging from less than a dollar to over USD60, attributing this wide variation in part 
to the lack of any standardised system of reporting. In Figure A6, we provide a basic 
calculation of cost per beneficiary for a subset of around 10 per cent of the projects 
in our portfolio. 48 We find that the average cost per benefidary is around USD 56, 
with the majority of values clustered around the USD20- 40 mark. 49 Some of the 
higher figures included here are likely to include start-up costs or costs of inputs 
such as seeds or tools, as well as variable costs, and so provide upper bound 
estimates on the costs of implementing and maintaining field schools. 



Note: Current costs not adjusted for inflation. 


Crops, livestock and curricula 50 

While the first FFS targeted rice farmers, they have increasingly been adapted for a 
wide variety of crops and livestock to reflect the diversity of farming practices 
around the world (Davis et al., 20 10) . Some FFS have supported the production of a 
number of different crops or types of livestock husbandry while others have focused 
more narrowly (see Figures A7 and A8). The large majority (92%) of portfolio 
projects developed FFS which targeted some form of crop, in particular cereal crops, 
root crops, vegetables, cash crops such as tea or coffee and fruit. Over a third of the 
projects collected in our portfolio supported some form of livestock farming, mainly 
poultry, cattle, and sheep and/ or goats. 


48 See Appendix Al, Data extraction and analysis (3). 

49 See Appendix Al, Data extraction and analysis (4) . 

50 See Appendix Al, Data extraction and analysis (5), for a note on how totals were calculated. 
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Figure A7 Percentage of projects addressing crop topics 



Notes: * Incorporates all crop types, tlncluding tea, coffee, cocoa, cotton, sugarcane etc. 

$ Such as potato, manioc, cassava etc. § Where a data source has indicated crops without 
specifying type. H Including floriculture, honey etc. Projects can cover multiple crops, hence 
total percentage sums to greater than 100 per cent. 


Figure A8 Percentage of projects addressing livestock 



Notes: * Incorporates all livestock types, t Where a data source has indicated 'livestock" 
without specifying type, f Such as boar, rabbits and other small animals. Projects can cover 
multiple livestock, hence total percentage may sum to greater than 100 per cent. 


The foundation of the FFS approach involves participatory learning in the field with 
farmer observations as the basis for analysis and learning (Gallagher, 2003). FFS 
sessions themseves are usually built around a flexible set of curriculum components 
which follow the subj ect topics of interest. Figure A9 provides a breakdown of the 
different curriculum and complementary components that are common elements of 
projects in the portfolio. Also depicted are the estimated share of projects which 
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included complementary upstream and downstream components such as input 
support, platform building and marketing support. A brief summary of each 
component is provided in Figure A10. 

Analysis of the portfolio shows that a high proportion of the projects included 
training of trainers, curriculum development and group formation (each around 
90%), all of which would be expected to be core elements of an FFS. 51 Over half of 
the projects also incorporated field days and agro- ecosystem analysis, while many 
also included farmer exchanges, special topics and group dynamics components. 
However, only one- quarter of the projects reported that they had incorporated 
"business as usual" control plots. Pontius et al. (2002) state that control plots should 
be a part of every FFS for comparison purposes. Even allowing for the fact that some 
projects may have included control plots, but not documented this, our analysis 
suggests that this key aspect of the approach is not always implemented. It was also 
common for projects to incorporate complementary components, with around half 
providing additional inputs such as seeds or tools, a third settingup fanner 
organisations and networks, and a further third providing marketing training. 


Figure A9 FFS curriculum and complementary components 



Note: Projects cover multiple curriculum components, therefore total percentage sums to 
greater than 100 per cent. 


Figure A10 FFS curriculum and complementary components 

Curriculum components: 

• Training of trainers: This entails providing theoretical and practical training to all FFS 
trainers. Training is usually season-long and typically includes facilitation and teaching 
skills, farming techniques and management skills. 

• Curriculum development: This involves planning a farmer field school's workplan, 
setting out the different modules to be studied, usually with some inbuilt flexibility. 

• Group formation: This involves setting up an organised group with specified goals such 


51 In interpreting these results it is important to bear in mind that not all the available documentation 
for the portfolio projects comprehensively reported on curricula The growth in the number of farmer 
field school interventions has also made it possible for some projects to recruit trained staff and borrow 
or adapt existing curricula In addition, some FFS have actively targeted pre- existing farmer groups. 
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as a shared workplan, and which has an organisational structure such as a representative 
committee. 

• Field day: Field days are designed to promote the work going on in an FFS by inviting 
local farmers and people of influence to see what has been learned and the benefits that 
can be produced. 

• Agro- ecosystem analysis ( AESA) : This involves undertaking regular observations of 
crops to understand the interrelationship between soil, the natural elements and 
biological pests or weeds. 

• Farmer exchanges: This involves visits by FFS members to other schools with a view to 
observing their activities and exchanging ideas. 

• Special topic: This curriculum component involves the selection of a topic which is 
usually either suggested by the fanners themselves or is chosen as a response to field 
observations (e.g. the discovery of a pest in a field). 

• Group dynamics: This activity is designed to develop leadership, problem- solving, 
discussion and communication skills. The emphasis is also on collaboration and the 
promotion of collective action. 

• Control plot: This involves the setting aside of control plots which either receive no 
treatment at all or continue to be farmed as before. These plots serve as a baseline 
comparison for those where new techniques or technologies are implemented. 

Other component: This category includes components specifically built for a particular 
group, such as hill farmers or ethnic minorities. Other additional components include 
community education and animal husbandry. 

Complementary components: 

• Platform building: This includes efforts to ensure that the benefits of FFS are sustained 
in the long term. Examples of such activities include organising farmers' clubs or 
building local networks which can ensure continued collective action. 

• Input supply: This entails the provision of fanners with seeds, tools or other equipment. 

• Marketing: This component involves providing farmers with training and help in setting 
up networks, transport and information- sharing geared towards marketing their 
products. 


One of the core elements of the vast maj ority of the proj ects examined in the 
portfolio was to provide training of trainers to produce a cadre of staff able to 
facilitate the FFS. FFS trainers do not instruct or provide lectures, but instead 
facilitate the learning process (Khisa, 2004) . Trainers may simply be local 
agricultural experts who have participated in a training course (extension trainers), 
but increasingly farmers who have themselves graduated from fanner field schools 
have also been encouraged to undertake training and take the lead role in facilitation 
(farmer trainers) (Figure All) . 
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Figure All Training of trainers 
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Documentation for around 20 per cent of the portfolio projects recorded this 
distinction between extension and fanner trainers. Data analysis of this sample 
showed that around 50 per cent of global FFS trainers have been farmer trainers. 
This trend was reflected in Asia, though was less true in Africa. Often, as in the case 
of the Indonesian National IPM Programme (Van den Berg et al., 2004), an initial 
cohort of extension trainers has helped to train later groups of farmer trainers. This 
is especially true of larger- scale proj ects with a longer duration. It may be that for 
African FFS a lower proportion of all trainers were farmer trainers because, on 
average, African FFS projects have been on a smaller scale than Asian ones. It is also 
worth noting that the distinction between types of trainer is more likely to have been 
documented for projects which encouraged the training of farmer trainers, with the 
consequence that this analysis may exaggerate their relative numbers. Analysis of 
the portfolio also indicates that FFS trainers were more than twice as likely to be 
male than female, regardless of region. 

Field school objectives 52 

The proliferation of variations in goals and focus of farmer field schools is reflected 
by the emergence of a variety of different types of FFS. These include those geared 
towards specific groups such as J unior Farmer Field and life Schools, which target 
orphans and youths between 12 and 18 years old who come from communities where 
HIV/ AIDS has impacted on food security. Other variants, such as business or 
marketing FFS, are intended to develop additional skills which can improve farmer 
livelihoods, or to adapt the original approach for an alternative type of farming, as 
with Farmer Livestock Schools. More generally, FFS have been designed to realise a 
number of more general goals. The categories in Figure A12 cover some of the most 
common objectives for FFS projects, as reported in project documentation. Figure 
A13 provides some definitions for each objective type. 


52 See Appendix Al, Data extraction and analysis (5), for a note on how totals were calculated. 
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Figure A12 Project objectives 



Note: Projects may have multiple objectives, therefore total percentage sums to greater than 
100 percent. 


Figure A13 Project objectives definitions 

• Production: This includes objectives such as increased production, increased quality of 
production and improved production techniques. 

• Empowerment: This objective refers to the desire to improve beneficiaries' self-regard, 
social skills and control over assets, as well as interactions with other local farmers, local 
government and service providers. 

• Market access: This involves providing training and help in setting up networks, 
transport and information- sharing geared towards marketing produce. 

• Food security: This involves improving access to sufficient, safe and nutritious food. 

• Environment: This category includes education on the environment and climate change, 
sustainable land and water use, the reduction of negative environmental impacts from 
farming and the protection of the local environment and existing natural assets through 
dyke construction or forestation/ reforestation, for example. 

• Social and community objectives: This includes objectives such as reducing gender 
inequality, targeting minority groups, community development or strengthening 
producer groups. 

• Institutionalisation: This includes the construction of infrastructure, or the setting up of 
networks or groups with the intention of ensuring ongoing FFS benefits. 

• Health: This includes targeting improved health outcomes for local populations through 
education or the reduction of harmful chemicals used in agriculture. 

• Pest, pesticide and soil management: This includes the introduction of pest control 
methods, reduced use of chemical pesticides and the implementation of crop or soil 
management techniques. 

• Animal healthcare: This category includes objectives such as improving animal health, 
husbandly and the provision of veterinary support. 


Improved production was a central goal for almost all the portfolio projects, 53 with 
the majority also aiming to boost food security, strengthen communities and 


53 A proj ect was only coded as targeting production where it was dearly stated as a goal or this was dear 
hum context in the documentation available. In reality, improved production is such a universal goal 
for FFS that the figure here may well underestimate the proportion of proj ects targeting it. 
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empower individuals. Protecting the environment or promoting sustainable farming 
techniques was also a very common obj ective as was some form of 
pest/ pesticide/ soil management (see Figure A14 for a more detailed analysis of FFS 
and pest management) . Around half of the proj ects targeted health improvements, 
market access and institutionalisation of FFS, while only a small minority were 
designed to support animal healthcare. 

The first farmer field schools in Asia were designed to introduce integrated pest 
management (IPM) to tackle the problems resulting from an over- reliance on 
chemical pesticides (Van den Berg &J iggins, 2007). Since then, farmer field schools 
have evolved so that many focus on different soil management or production 
techniques rather than on pests. Around 60 per cent of the portfolio projects 
explicitly stated that some form or combination of pest, pesticide or soil 
management was an objective (although this type of component is such a common 
element of the FFS approach that it is likely that some projects included such 
components, but details were not documented) . A large proportion of the projects 
that did report this type of component stated that IPM had been a part of the FFS. A 
significant number of projects have also pursued variant forms of IPM, developed to 
suit specific geographical locations or objectives. Figure A14 sets out the incidence of 
the different management techniques as reported. 

The majority of projects that reported on this type of management technique stated 
that IPM was incorporated as part of the FFS intervention. Variant integrated 
techniques such as IPPM, I CM and ICPM were also recorded, while others simply 
reported some form of pesticide management or soil management (see Figure A15 
for definitions). In many cases, the type of pest, pesticide or soil management 
utilised is reported only briefly. Where more detail is available, there is considerable 
overlap between the usage of the different concepts. While some of the 
categorisations below are fairly clearly defined in FFS literature, others are more 
general terms whose meaning has been inferred from their usage. 

Figure A14 Pest, pesticide and soil management (percentage of projects) 


management 
.4%) 



ICM 
(4.0%) 

ICPM (2.2%) V Other (2.7%) 


I Integrated Pest Management 
(IPM) 

l Other pesticide management 

l Soil management 

Integrated Production and Pest 
Management (IPPM) 

Integrated Crop Management 
(ICM) 

Other 

Integrated Crop and Pest 
Management (ICPM) 


226 



Figure A15 Pest, pesticide and soil management definitions 

• Integrated pest management (IPM) : IPM draws on an understanding of the life cycles 
of pests and their interaction with the environment to manage populations of pests 
economically while minimising risks to the environment or human health. 

• Other pesticide management: This category includes non- specific references to 
chemical or pesticide management techniques. 

• Soil management: This category includes non-specific references to soil or crop 
management techniques. 

• Integrated production and pest management (IPPM): 1 PPM is a variant of I FM which 
has eveolved in Africa and which emphasises both the management of natural pests 
and the production of a healthy crop. 

• Integrated crop management (ICM) : ICM draws on an understanding of interactions 
between soil, the natural environment and biological pests or weeds to promote 
sustainable crop production. Example components include site selection, crop- specific 
production strategies, nutrient management and cover cropping. 

• Other: This category includes other variants of IPM (examples include integrated pest 
and vector management [IPVM] or integrated pest biosystem management [IPBM]), or 
other general references to management techniques. 

• Integrated crop and pest management (ICPM): ICPM combines chemical, biological 
and cultural pest control methods with crop management strategies. 


Participant groups 54 

Figure A16 sets out some of the different participant groups and themes that FFS 
commonly address. Around a quarter of the projects supported producer groups 
with smaller proportions including pre- existing farmer associations and pastoralists. 
A significant minority of the projects promoted positive outcomes for forestry or 
agro- forestry. One aspect of the farmer field schools approach that is often put 
forward as an important strength is its ability to reach out to marginalised or 
minority groups who might otherwise not have access to the training, knowledge, 
employment or inputs. Over half of all projects included women, while just under a 
quarter targeted youths and around 12 per cent targeted HIV/ AIDS sufferers. 
Smaller numbers included displaced people, orphans, ethnic minorities and the 
elderly. 

Despite the fact that women comprise, on average, 43 per cent of the agricultural 
labour force in developing countries, they have far less access than men to 
productive resources and opportunities (FAO, 2011). Agricultural extension 
programmes such as farmer field schools have been put forward as a potential way 
to support female farmers, with some projects specifically targeted at women and 
others designed to be inclusive. 


54 See Appendix Al, Data extraction and analysis (5), for a note on how totals were calculated. 
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Figure A16 Participant groups and themes 



Note: Projects may target multiple groups and themes, therefore total percentage sums to 
greater than 100 per cent. 

Although over half of the projects in the portfolio reported that they had either 
included, or explicitly set out to target, female farmers (see Figure A16), there is 
some evidence to show that gender equity elements in FFS proj ects have been more 
successful, or at least more heavily implemented, in Africa. Indeed, while global 
totals based on the portfolio show that FFS- benefiting farmers were more likely to 
bemale(70%) than female (30%), women made up the majority of FFS beneficiaries 
in Africa. This emphasis may be, in part, a result of FFS being seen as a way to 
overcome the traditional problems of extension in reaching and empowering female 
farmers in Africa (see, for example, Saito &Weidemann, 1990; World Bank & IBRD, 
2009) . It may also reflect the predominant role that African women play in 
agriculture (Udiy et al., 1995). Only a small proportion (around a third) of the 
proj ects in the portfolio actually report numbers of benefiting farmers subdivided by 
gender. It may be that those proj ects reporting figures for women do so precisely 
because of their emphasis on female empowerment, with the result that the figures 
set out in Figure A17 actually overestimate female inclusion. 


228 




Figure A17 Gender balance among participants 
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APPENDIX A1 PORTFOLIO REVIEW RESEARCH 
METHODOLOGY 


Search methodology 

Relevant documentation was identified through an internet search, a review of 
studies already included in the effectiveness review of this report, documentation 
provided directly by IFAD and documentation provided directly by expert contacts. 

The internet search was based on ex- ante identification of appropriate websites and 
search terms. Searches were carried out in the following electronic databases: 
EconLit (EBSCO); IDEAS/ RePEc repository; Science Direct; International 
Bibliography of the Social Sciences - IBSS (EBSCO); J STOR; J OLIS; Zetoc (British 
Library); BLDS and ELDIS (portals); Mendeley. 

In addition to these databases, the following agriculture- and FFS- related sources 
were also searched: Global FFS- Net / FFS Foundation; LEI (Agricultural Economics 
Research Institute); DAC (Development Assistance Committee) evaluation database; 
ODI (Overseas Development Institute); NRI (Natural Resources Institute); CABI 
(Centre for Agricultural Biosdence International); ILEIA Newsletter (Centre for 
Learning on Sustainable Agriculture); FAO, especially documents contained in the 
FAO Document Repository, 55 the FAO IPM Near East website 56 and the FAO IPM 
vegetable programme in Asia; 57 IFAD, especially documents held on the IFAD 
projects by country page 58 and the IFAD intranet "xdesk". 

Combinations of the following set of search terms were used: 

"Farmer field school"-!- 


55 http:/ / www.fao.org/ documents/ 

56 http://vww.ipm-neareast.com 

57 http:/ / www.vegetableipmasia.org/ Concepts.html 

58 http:/ / wvvw.ifad.org/ operations/ proj ects/ regions/ countrv.htm 
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a) Geographical coverage 

• name of country ( all countries were searched for) 

• name of continent ( all continents were searched for) 

b) Funders 

• "FAO", 'IFAD" "EU", "World Bank", "UNDP", "Asian Development Bank", 

"African Development Bank", "IFPRI", name of main foreign 

government's department for international development 

c) Type of document 

• Appraisal report. Final report. Activities report. Annual report. Quarterly 
report. Progress report. Terminal report. Completion report. Evaluation report 

All possible combinations of the different terms were searched for in order to ensure 
that content was captured as comprehensively as possible from the different 
databases. Follow-up searches on citations found in studies retrieved from the initial 
round of searches were also carried out. In all, the search strategy returned a total of 
1,916 documents, including evaluations, project documents, project descriptions, 
academic articles, grey literature and personal communications. A list of all included 
texts is provided in Appendix A2. 

Inclusion criteria 

Only documentation that described projects involving farmer field schools was 
included in the portfolio analysis. Projects that only involved other agricultural 
extension activities were excluded. Additionally, projects were only included in the 
portfolio if they met one or more of the following criteria: 

1. Documentation provided data on one or more of the following: the number of 
fanner field schools implemented by a proj ect, the number of benefiting fanners, 
the number of FFS trainers. 

2 . Documentation provided information on intervention components ( such as 
training of trainers, curriculum development etc.). 

3. Documentation provided data on the total cost of the proj ect or the cost of 
implementing FFS- specific components. 


Data extraction and analysis 

Two independent reviewers assessed the papers against the inclusion criteria, with 
any discrepancies resolved through discussion. Each author extracted data from a 
subset of the included studies, with data coded in Excel. 

Entries were made by FFS projects, with data from all relevant documentation for a 
particular programme included as a single entry. Analysis of coded data for the 337 
programmes captured by the portfolio review was then carried out in Excel and 
Stata. 

( 1) Figures for both the number of FFS implemented and the number of 

benefiting farmers were available for just over 60 per cent of the 260 projects 
in the portfolio. This subset of portfolio projects was examined to discover how 
many beneficiaries, on average, graduated from each FFS. Both the average 
and the modal value for the beneficiaries from an FFS equalled 25, a figure 
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also common in the FFS literature. By assuming that one FFS produces 25 
graduating farmers for those proj ects for which only partial information was 
available ( either number of benefiting farmers or number of FFS) , we were 
able to estimate the numbers of FFS and graduating farmers for 97 per cent of 
the portfolio projects. 

(2) In order to combine the figures from our portfolio with those from the Braun 
et al. (2006) dataset, the two datasets were first compared to ensure that, 
where possible, figures were not double- counted. The following steps were 
then undertaken to calculate lower and upper estimates: 

• Lower estimates were calculated by adding figures from the Braun et al. 
(2006) dataset only where they were higher than those already included 
in the portfolio for a particular country. Where Braun et al. (2006) had 
data on a particular country and the portfolio did not, or vice versa, these 
figures were included in the total. 

• Upper estimates were calculated by adding all figures from the Braun et 
al. (2006) dataset to those from the portfolio. 

(3) It was not possible to calculate cost per beneficiary for the maj ority of the 
portfolio, either because of insufficient information or because they 
constituted large agricultural or developmental programmes for which the 
implementation of FFS methodology is only one element within a far larger 
whole. 

(4) This analysis included only either (a) projects that reported cost per 
benefidary or (b) proj ects for which both the number of benefidaries and the 
cost of FFS- related components were available. For the latter category, cost 
per benefidary was calculated by dividing total cost by number of farmers 
trained. 

( 5) Analysis reported in the portfolio for Crops, livestock and curricula. 

Objectives and Partidpant groups was coded based on a document providing 
information on either, the intention to indude a particular component, 
objective or partidpant group, or a report of its actual indusion in a 
programme. 


APPENDIX A2 PORTFOLIO INCLUDED STUDIES 
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Appendix B 


EXAMPLE SEARCH STRATEGY 


Farmer Field Schools Review Update (August- October 2012): CAB 

Abstracts (Ovid) 

1. (integrated control or integrated pest management or crop management). sh. 

2. ( (integrated adj (production or management or pest or nutrient) ) or crop 
management) .ti,ab. 

3. 1 or 2 

4. on-farm training.sh. 

5. (field school* or farm* school* or farmer* field*). ti,ab. 

6. (practical education or extension education or education programmes or 
community education or agricultural education or inservice training or vocational 
training or innovation adoption) .sh. 

7. extension/ 

8. (participatory extension or agricultural advisory or agricultural extension or 
rural extension) .ti,ab. 

9. 6 or 7 or 8 

10. exp developing countries/ or exp afiica/ or exp asia/ or exp south america/ or 
exp central america/ or exp latin america/ or exp pacific islands/ or exp middle 
east/ or mexico/ 

11. 3 and 9 and 10 

12. 5 and 9 and 10 

13. 4 or 11 or 12 

14. (farmer* adj field* adj school*). ti,ab. 

15. 10 and 14 

16. 13 or 15 
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RECORD OF DATABASE SEARCHES 


Name of database 

Date of search 

International Bibliography of Social Sciences (ProQuest) 

May 2010; Updated: 9 October 2012 

EconLit (EBSCO host) 

May 2010; Updated 2 August 2012 

Web of Knowledge (Social Sciences Citation Index and 
Social Science Conference Proceedings) 

May 2010; Updated 15 August 2012 

AgEcon 

May 2010; Updated 2 August 2012 

CAB Abstracts 

June 2010; Updated 2 August 2012 

EBSCO multifile search - Academic Search Research and 
Development, Africa-Wide Information, Business Source 
Complete, Soclndex 

19 October 2012 

United States National Agricultural Library (Agricola) 

May 2010 

ProQuest: Dissertations and Theses 

May 2010 

Index to Theses 

July 2010 

Theses Canada 

July 2010 

BLDS 

May 2010 

JOLIS 

May 2010 

IDEAS 

May 2010 
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RECORD OF INTERNET SEARCHES 


Organisation 

Website 

Search date 

African Development Bank 
(AfDB) 

http://www.afdb.ora 

02-Jul-10 

Agricultural program IFAP 
Asian Development Bank 
Australian Aid Agency 
Canadian international 
development agency 
Centre for Environment and 
Society (Essex University) 
CGI AR research centres 
(including IFPRI, 

http:/ / www.ifap.ora * 
www.adb.ora 
www.ausaid.aov.au * 

www.acdi- dda.ac.ca 
http:/ / www.essex.ac.uk/ ces/ 

05-Jan-10 

ll-Jun-10 

ll-Jun-10 

ll-Jun-10 

05-Jan-10 



http: / / www.caiar.ora/ publications/ library/ 

12-Jul-10 

International Potato Centre, 
QMMYT, IRRI, IWMI) 

index.html 


Christian Aid 

www.christianaid.ora.uk 

30-Jun-10 

QP- UPWARD (International 
Potato Center/ Users' 
Perspectives with 
Agricultural Research and 
Development) 

www.dp-upward.ora * 

ll-Jun-10 

Cotton IPM Asia 

www.cottonipmasia.ora (website not 
workina) 

06-Jan-10 

Danish development agency 

www.um.dk 

ll-Jun-10 

Environment for 
Development (EfD) 

www.efdinitiative.ora 

ll-Jun-10 

European Commission 

httD://ec.eurom.eu/ index en.htm 

12-Jul-10 

FAO (Food and Agriculture 
Organization of the United 
Nations) 

httx>:/ / www.fao.ora/ nr/ research/ informres 
/research- publications/ en/ * 

13- Aug- 10 

FaimAfnca 

www.farmafrica.ora.uk 

09-Jul-10 

Future Agricultures 

www. future- aaricultures . ora 

30-Jun-10 

GIZ (German international 
development agency) 

www.aiz.de/en 

09-Jul-10 

Global FFS Network and 
Resource Centre (FFS net) 

http: / / www.farmerfieldschool.info/ 

05-Jan-10 

Global Forum for Rural 
Advisory Services (GFRAS) 

http://www.a-fras.ora/ 

09-Jul-10 

Global IPM Facility 

Unable to locate website 

06-Jan-10 

Independent Science and 
Partnership Council (CGIAR) 

http: / / impact.caiar.ora/ 

06-Jan-10 

Inmasp 

http: / / www.lei.dlo.nl/ inmasp/ results.php3 

* 

06-J an- 10 

Inter- American Development 
Bank (IDB) 

http: / / www.iadb.ora/ 

ll-Jun-10 

International Development 
Research Centre (IDRC) 

http://www.idrc.ca/ en/ev- 1-20 1-1- 
DO TOPIC.html * 

05-Jan-10 

International Fund for 
Agricultural Development 
(IFAD) 

J apan International 

http:/ / www.ifad.ora 

05-Jan-10 

Cooperation Agency and 
J apan Bank for international 

www.iica.ao.ip/ enalish/ 
www.ibic.ao.ip 

12-Jul-10 

Cooperation 
Leibniz University of 
Hannover, Pesticide Policy 
Project 

http:/ / www.ifab.uni- 
hannover.de/ 2699.html * 

30-Jun-10 
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ODI Agricultural Research 

http://www.odi.om.uk/ networks/ aaren/ 


and Extension Network 

htto:/ / www.odi.ora.uk/ networks/ aaren/ du 

05-Jan-10 

(Agren) 

blistl.html * 


OECD-DAC 

http: / / www. dac- evaluations- 
cacLora/ abstracts e.htm 

09-Jul- 10 

Oxfam 

http:/ / publications.oxfam.ora.uk/ index.asp 

03-Jul- 10 

Poverty Action Lab 

www.povertvactionlab.ora 

Contacted key informant: operate a 
participatory extension programme, but no 

05-Jan-10 

Practical Action 

prospective evaluations with comparison 
group observations have been conducted to- 
date. 

U-Jun-10 

SdDevNet 

www.sddev.net 

09-Jul- 10 

Swedish International 
Development Cooperation 
Agency 

www.sida.ora 

ll-Jun-10 

Systemwide Programme on 
IPM 

htto:/ / www.SDinm.caiar.ora/ home 

06-Jan-10 

The World Bank -fOffice of 
Evaluation and Development 

www.worldbank.ora 

ll-Jun-10 

UK Department for 
International Development 

www. dfid. aov.uk 

09-Jul- 10 

US Agency for International 
Development 

www.usaid.aov 

12-Jul-10 


Note: * website no longer active as of J une 20 14. 
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JOURNALS HANDSEARCHED 


J ournal searched 

American Economic J oumal: Applied Economics 

Agricultural Economics 

American J oumal of Agricultural Economics 

Agricultural Economics Review 

J oumal of Agricultural Education 

J oumal of Agricultural Education and Extension 

J oumal of Extension 

J oumal of International Agricultural and Extension 
Education (USA) 

Integrated Pest Management Reviews 
J oumal of Sustainable Agriculture 
J oumal of Extension Systems (India) 

J oumal of Agricultural &Food Information 

AgBioForum 

LEISA 

Indian J oumal of Extension Education 
Indian J oumal of Agricultural Economics 
J oumal of Environmental Extension 
J oumal of Development Economics 
J oumal of Development Effectiveness 
J oumal of Development Studies 
J oumal of International Development 
Pest Management Science 
World Development 


Volumes 

Vol. 1(1), Vol. 3(1) 

Vol. 22(1), Vol. 42(2) 

Vol. 82(1), Vol. 93(1) 

Vol. 1(1), Vol. 11(2) 

Vol. 41(1)- Vol. 51(4) 

Vol. 7(l)-Vol. 17(1) 

Vol. 38- Vol. 49 Feb 
Vol. 7(l)-Vol. 17(3) 

Vol. 5(1)- Vol. 7(4) 

Vol. 15(4)- Vol. 35(2) 

Vol. 16- Vol. 25 
Vol. 4(1)- Vol. 12(1) 

Vol. 12(364), 2009 Special Issue 

Vol. 19(1), 2003 Special Issue: Leamingwith 

FFS 

Vol. 6(3)- Vol. 10(3) 

Jan 2000- Jul 2010 
Vol. l(2000)-Vol. 8 (2009) 

Vol. 61(1)- Vol. 95(1) 

Vol. 1(1)- Vol. 2(2) 

Vol. 36(3)- Vol. 47(2) 

Vol. 12(4)- Vol. 23(2) 

Vol. 56(1)- Vol. 67(4) 

Vol. 28(1)- Vol. 39(5) 
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REASONS FOR EXCLUSION OF MARGINAL STUDIES 

Effectiveness synthesis 



Study 

Reason for exclusion 

Comment 

Armen et al., 2002 

Methodology 

No non-FFS comparison 
group. 

Cole etal., 2002 

Methodology 

No non-FFS comparison 
group. 

Clavijo Ponce et al., 2007 

Relevabnce 

Not an intemvention study, 
evaluates farmers' 
knowledge. 

Dalton et al., 2005 

Relevance 

Q AL participatory research, 
notFFS 

Douthwaite et al., 2007 

Methodology 

No non-FFS comparison 
group. 

Duveskogetal., 2003 

Methodology 

No non-FFS comparison 
group, preliminary 
evaluation findings. 

Gambyetal., 2002 

Methodology 

No non-FFS comparison 
group. 

Iqbal etal., 2012 

Methodology 

No non-FFS comparison 
group. 

Heongetal., 1998 

Methodology 

No non-FFS comparison 
group. 

Kassieetal., 2011 

Relevance 

Not an intervention study; 
measures the impact of 
adoption. 

Katrak, 2006 

Methodology 

No non-FFS comparison 
group. 

Mancini et al., 2008 

Methodology 

No non-FFS comparison 
group. 

Nabiiye et al., 2003 

Relevance 

Participatory efficacy trial 
assessing IPM techniques. 

Ne Dort et al., 2006 

Methodology 

No non-FFS comparison 
group. 

Owenya et al., 2012 

Methodology 

No non-FFS comparison 
group. 

Pretty etal., 2011 

Relevance 

Not an impact study; review 
of FFS experiences in some 
countries, methodologies for 
case studies not reported. 

Rwegasira et al., 2004 

Relevance 

Not a study of effects. 

Samiayyan et al., 2010 

Relevance 

Not an intervention study; 
measures impact of 
adoption. 

Smale et al. 

Relevance 

Not FFS. 

Tin et al., 2010 

Methodology 

No non-FFS comparison 
group. 
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Tshiebue, 2010 

Relevance 

French language study; not 
an impact evaluation. 

Vattaetal., 2009 

Relevance 

Not FFS. 

Van den Berget al., 2004 

Methodology 

No non-intervention 
comparison, selected case 
studies. 

Van den Berget al., 2007 

Relevance 

Integrated vector 
management study 

Wijeratne et al., 2004 

Methodology 

No non- FFS comparison 



group. 

Yajima, 2010 

Methodology 

No non-intervention 



comparison. 

Qualitative synthesis 

Study 

Reason for exclusion 

Comment. 

Afreh- Nuamah, 2003 

Methodology and relevance 

Mainly a review of 
experience so far, with one 
case study. The case study is 
an efficacy trial. 

Anandajayasekeram et al.. 

Methodology 

Limited description of 

2007 


methods. 

Assefa and Williamson, 2001 

Methodology 

Limited description of 
methods, not even clear it is 
a primary study. 

Banu and Bode, 2003 

Methodology 

No description of methods 

Bartlett, 2005 

Methodology 

Not dear whether primary 
study. Lack of information 
on methods of data 
collection, sampling and 
sample characteristics. 

Barzman and Desilles, 2001 

Relevance 

No assessment of barriers or 
enablers. 

Beckmann et al., 2006 

Methodology and relevance 

Not clear whether FFS and 
no information on methods. 

Beider et al., 2006 

Methodology and relevance 

Not clear whether primary 
study. Lack of detail on 
methods. 

Bijlmakers and Islam, 2008 

Methodology 

No description of methods, 
data collection, sampling or 
sample characteristics. 

Bokor, 2001 

Methodology 

No information on sampling 
and sampling characteristics. 

Bunch, 2003 

Methodology 

No details on methods, not 
clear if primary study. 

Bwalya, 2005 

Methodology 

Lack of information on 
methods. 

Chianu &Tsujii, 2005 

Relevance 

Not addressing barriers and 
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CORAD, 2008 

Din &Morisson, 2003 
Duveskogetal., 2002 
Fakih et al., 2003 

Fakih, 2003 
FAO, 2008 

Gallagher, 2000 

Gallagher, 2003 
Gill, 2004 

Goff et al., 2009 

J alalzadeh et al., 2009 
Kebebe et al., 2007 

Khisa and Heinemann, 2005 
Khisa, 2003 


Relevance 

Methodology 

Methodology 

Methodology 

Methodology 

Methodology 

Relevance 

Methodology 

Methodology 

Methodology and relevance 
Methodology 

Methodology and relevance 

Methodology 

Methodology 


facilitators. The focus is on 
efficacy of various INM 
practices. 

Some information on 
targeting and participation, 
but no information on 
barriers and facilitators. 
Insufficient information 
provided on sampling and 
sample size. 

No description of data 
collection, sampling, no dear 
research question. 

Lack of information about 
methods and no data 
presented to support the 
findings / condusions. 

No description of methods, 
data collection, sampling or 
sample characteristics. 

No systematic description of 
methods, sampling and 
recruitment or sample 
characteristics. 

Paper does not focus on 
specific case, and not dear 
there is any primary data 
collection. 

Not primary study. 

Not clear whether primary 
study, no information on 
sampling, sampling 
characteristics, research 
question. 

Limited information on 
barriers and fadlitators, 
limited information on 
sampling. 

Methods not dear, limited 
information on barriers and 
facilitators 
Efficacy study. No 
information provided on 
methods (sampling, sample, 
data collection). 

No information on data 
collection or sampling. 

No description of methods. 
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Kishi, 2002 


Lopez Gaytan et al., 2008 
Makhdum et al., 2003 
Makhdum et al. 2002 
Mallah &Kerejo, 2007 

Nathaniels., 2005 

Okoth et al., 2002a 

Okoth et al., 2002b 
Pontius, 2003 


Price and Gurung, 2006 

Prudent et al., 2007 

Rahman, 2012 

Schut and Sherwood, 2007 

Settle &Garba, 2011 


Methodology 

Methodology 

Methodology 

Methodology 

Methodology 

Methodology 

Methodology 

Methodology 

Relevance 


Methodology and relevance 

Methodology 

Methodology 

Methodology 

Methodology and relevance 


Insufficient information on 
sampling and no sample 
characteristics presented. 
Insufficient information on 
sampling. 

Lack of information on 
sampling. 

No description of methods. 
No information on sampling. 
Lack of reporting on 
methods, including sampling 
and sample characteristics. 
Insufficient information 
provided on sampling 
procedures and sample 
characteristics. 

Not clear if any primary 
research, insufficient 
methods reporting. 

Not clear if primary data, no 
details on methods. 

Looks at the impact of 
Farmer study groups 
(Community I PM) and does 
not assess 

barriers/ facilitators. 
Insufficient reporting of 
methods (no info on 
sampling and sample 
characteristics). 

Comparative study focusing 
on comparative methods and 
outcomes. Sample selection 
criteria not entirely clear and 
insufficient information on 
sample characteristics. 
Insufficient information on 
methods, no information on 
sample, limited information 
on data collection. 

Lack of info on methods: no 
information on sample 
characteristics. 

Insufficient information 
provided on data collection, 
no information on sampling 
and sample characteristics. 
Not clear whether primary 
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study, no reporting on 
methods. No focus on 
barriers and facilitators. 

Simpson and Owens, 2002 

Methodology 

Limited description of 
methods. 

Stock, 1995 

Methodology 

Lack of info on methods: no 
info on sampling or sample 
characteristics. 

Tengetal., 2005 

Methodology 

Not clear if primary study 
and insufficient information 
of methods. 

Van Beek et al., 2004 

Methodology 

Lack of detail on sampling. 

Van den Berget al., 2006 

Methodology 

Undear whether primary 
study. Not enough 
information provided on 
methods induding sampling 
and sample charaderistics. 

Van den Berget al., 2007 

Relevance 

No information on barriers 
and fadlitators. Limited 
information on methods. 

Wamaars and Pradel, 2007 

Methodology 

No information on sampling 
and recruitment. 

Williamson et al., 2003 

Methodology 

Limited description of 
methods, insuffident 
information on sample 
characteristics. 

Winarto, 2000 

Methodology 

Not dear if primary research, 
no information about 
methods. 
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DATA COLLECTION CODES 


Quantitative impact evaluations (studies addressing review question 1) 


Intervention design 

Intervention type: IPM, I PPM, IPNM, IWM, other 
Components of intervention 

Additional interventions provided, e.g. input support, 
marketing support 

Intervention period (from MM/YY to MM/YY) 

Context 

Country and region: East Asia & Pacific (EAP), Latin America 
& Caribbean (LAC), Middle East & North Africa (MENA), 
South Asia ( SA), sub- Saharan Africa ( SSA) 

Crops: cotton, rice, vegetables, other 
Length of follow-up (months) 

Study design 

Study type: RCT, quasi- RCT, RDD, natural experiment, DID, 

IV, ITS, PSM, adjusted (multivariate) single difference 

regression, unadjusted comparison of means 

Description of comparison group (and if relevant non- 

participant neighbour group) intervention 

Period of outcomes data collection (from MM/YY to MM/YY) 

Frequency of outcomes data collection 

Information reported on method of allocating individuals to 

groups 

Sample size (treatment, exposed, comparison): number of 
dusters, number of individuals 

Sample attrition (treatment, exposed, comparison), if relevant 
Spillovers: geographical separation of treatment and 
comparison 

Contamination: influence of other intervention which 
differentially affects treatment and comparison groups on 
relevant outcomes 

Risk of bias assessment (see Appendix 2) 

Effect estimation 

Treatment effect estimated: intention-to- treat (ITT), average 
treatment effect on the treated (ATET), average treatment 
effect (ATE), local average treatment effect (LATE) 

Adjusted or unadjusted analysis. 

Intermediate outcomes 

Knowledge: e.g. knowledge of "simple", "intermediate" and 
"complex" practices 

Adoption: e.g. use of pestidde, cost of pestidde, number of 
"improved" practices 

Final outcomes 

Yidds: weight per unit of land 

Net revenue: value of production minus costs of production 
(farm income or profits) per unit land 
Environment: e.g. environmental impad quotient score 
Health: e.g. inddence of health complaint (eye irritation, 
respiratory problems, stomach ache) 

Empowerment: e.g. reported self-esteem 
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Qualitative evaluations (studies addressing review question 2) 


# 

ID 

Description 

Coding 

1 

Author 

First author 

Open answer 

2 

Country 

Country 

Open answer 

3 

Facilitators 

Evidence on barriers and enablers related to 
the farmer field school facilitators, their 
recruitment, training, characteristics, 
support, relationship with participants etc. 

Open answer 

4 

Participants, 
targeting, equity 

Evidence on barriers and enablers related to 
the participants, including targeting and 
recruitment, group composition (including 
equity), participant characteristics and 
attitudes, attendance and drop-out rates etc. 

Open answer 

5 

FFS content and 
coverage 

Evidence on barriers and enablers related to 
content and coverage of farmer field 
schools, including appropriateness and 
relevance of curriculum, breadth and depth 
of coverage of topics, complexity of learning 
material etc. 

Open answer 

6 

Complexity 

Evidence on barriers and enablers related to 
complexity of the FFS curriculum / learning 
approaches 

Open answer 

7 

Observability 

Evidence on barriers and enablers related to 
observability of the techniques promoted in 
FFS, and their results 

Open answer 

8 

Relative advantage 

Evidence on barriers and enablers related to 
the (perceived) relative advantage of the 
FFS- promoted methods and approaches 

Open answer 

9 

Diffusion to non- 1 PM 
fanners 

Evidence on barriers and enablers related to 
diffusion of FFS practices to non- 1 PM 
farmers, such as the role of social capital, 
complexity and observability of the 
techniques and learning materials, 
observability of the benefits of the FFS 
approach etc. 

Open answer 

10 

Language/ discourse 

Evidence on barriers and enablers related to 
the language or discourse used in FFS 
training 

Open answer 

11 

Delivery 

(participatory vs. top- 
down, 

experimentation) 

Evidence on barriers and enablers related to 
the use, or lack thereof, of the participatory, 
experimental learning approaches 

Open answer 

12 

Service selivery / 
implementation 

Evidence on barriers and enablers related to 
the service delivery and implementation of 
the intervention, such as availability and 
timeliness of resources, appropriateness and 
accessibility of FFS location, timing of 

Open answer 
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# 

ID 

Description 

Coding 



trainings, follow-up etc. 


13 

Access to inputs, 
labour, markets 

Evidence on barriers and enablers related to 
farmers' access to inputs, labour, linkages to 
markets etc. 

Open answer 

14 

Community / context 

Evidence on barriers and enablers related to 
the community context and other (non- 
policy) contextual factors 

Open answer 

15 

Policy context 

Evidence on barriers and enablers related to 
the policy context where FFSs were 
implemented 

Open answer 

16 

Institutional set-up 

Evidence on barriers and enablers related to 
the international, national, regional and 
local institutional arrangements for 
implementing FFS 

Open answer 

17 

Gender and 
empowerment 

Evidence reported on the perceived effects 
of FFS on empowerment and gender roles 

Open answer 

18 

Sustainability 

Evidence on the barriers to and enablers of 
sustainability of FFS practices and outcomes 

Open answer 

19 

Other 

Evidence on any other relevant barriers and 
enablers noted in the study 

Open answer 
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Appendix C 


CRITICAL APPRAISAL METHODS 


Review question (1): Risk of bias assessment for impact evaluations 39 

Studies were critically appraised according to the likely risk of bias based on: 1) 
quality of attribution methods (addressing confounding and sample selection bias); 
2) the extent of spillovers to fanners in comparison groups; 60 3) outcome and 
analysis reporting bias; and 4) other sources of bias. "Low- risk-of- bias" studies are 
those in which clear measurement of and control for confounding was made, 
including selection bias, where intervention and comparison groups were described 
adequately (in respect of the nature of the interventions being received) and risks of 
spillovers or contamination were small, and where reporting biases and other 
sources of bias were unlikely. Studies are identified as at "medium risk of bias" 
where there are threats to the validity of the attribution methodology, or likely risks 
of spillovers or contamination, arising from inadequate description of intervention 
or comparison groups or possibilities for interaction between groups such as when 
they are from the same community, or possible reporting biases. "High- risk-of- bias 
studies" are all the others, including those where comparison groups are not 
matched or differences in covariates are not accounted for in multivariate analysis, 
or where there is evidence for spillovers or contamination to comparison groups 
from the same communities, and reporting biases are evident. Detailed risk of bias 
evaluation criteria are presented below. 

1) Selection bias and confounding 

a) For randomised assignment (RCTs) 

Score "YES" if: 

• a random component in the sequence generation process is described ( e.g. 
referring to a random number table); 

• and if the unit of allocation was at group level ( geographical/ social/ 
institutional unit) and allocation was performed on all units at the start of 
the study; 

• or if the unit of allocation was by beneficiary or group and there was some 
form of centralised allocation mechanism such as an on-site computer 
system; 


59 Wedrewon3ie (n.d.) and EPOC (n.d.) in developing this tool. 

60 Note that, in contrast, spillovers to "exposed" fanners are desirable for the intervention, and will be 
assessed by the measured effects reported on these groups, in separate meta- analysis. 


257 


• and if the unit of allocation is based on a sufficiently large sample size to 
equate groups on average; 

• and if baseline characteristics of the study and control/ comparisons are 
reported and overall similar based on t-test or ANOVA for equality of means 
across groups; 

• or covariate differences are controlled using multivariate analysis; 

• and the attrition rates (losses to follow-up) are sufficiently low and similar in 
treatment and control; 

• or the study assesses that loss to follow-up units are random draws from the 
sample (e.g. by examining correlation with determinants of outcomes, in 
both treatment and comparison groups); 

• and problems with cross- overs and drop- outs are dealt with using intention- 
to- treat analysis or in the case of drop-outs, by assessing whether the drop- 
outs are random draws from the population; 

• and, for cluster assignment, authors control for external cluster- level factors 
that might confound the impact of the programme (e.g. weather, 
infrastructure, community fixed effects etc.) through multivariate analysis. 

Score "UNCLEAR" if : 

• the paper does not provide details on the randomisation process, or uses a 
quasi- randomisation process for which it is not dear whether it has 
generated allocations equivalent to true randomisation; 

• or insuffident details are provided on covariate differences or methods of 
adjustment; 

• or insuffident details are provided on duster controls. 

Score "NO" if: 

• the sample size is not suffident; 

• or any failure in the allocation mechanism or execution of the method could 
affect the randomisation process. 

b) For discontinuity assignment (regression discontinuity design) 

Score "YES" if: 

• allocation is made based on a predetermined discontinuity on a continuous 
variable (regression discontinuity design) and blinded to partidpants; 

• or if not blinded, individuals reasonably cannot affed the assignment 
variable in response to knowledge of the partidpation dedsion rule; 

• and the sample size immediately at both sides of the cut- off point is 
suffidently large to equate groups on average; 

• and the interval for selection of treatment and control group is reasonably 
small; 

• or authors have weighted the matches on their distance to the cut- off point; 

• and the mean of the covariates of the individuals immediatdy at both sides of 
the cut-off point (selected sample of partidpants and non- partidpants) are 
overall not statistically different based on t-test or ANOVA for equality of 
means; 

• or significant differences have been controlled for in multivariate analysis; 

• and, for duster assignment, authors control for external cluster- level factors 
that might confound the impact of the programme (e.g. weather, 
infrastructure, community fixed effects etc.) through multivariate analysis. 

Score "UNCLEAR" if: 

• the assignment variable is either non- blinded or it is unclear whether 
partidpants can affect it in response to knowledge of the allocation 
mechanism; 

• or there are covariate differences across individuals at both sides of the 
discontinuity which have not been controlled for using multivariate analysis; 

• or if insuffident details are provided on (cluster) controls. 

Score "NO" if: 
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• the sample size is not sufficient; 

• or there is evidence that participants altered the assignment variable prior to 
assignment. 

c) For identification based on an instrumental variable (IV estimation) 

Score "YES" if: 

• an appropriate instrumental variable is used which is exogenously 
generated: e.g. due to a "natural" experiment or random allocation. 

• the instrumenting equation is significant at the level of F> 10 (or if an F test 
is not reported, the authors report and assess whether the R- squared 
(goodness of fit) of the participation equation is sufficient for appropriate 
identification); 

• the identifying instruments are individually significant (p < o . o 1) ; for 
Heckman models, the identifiers are reported and significant (p<o.os); 

• where at least two instruments are used, the authors report on an over- 
identifying test (p< 0.05 is required to reject the null hypothesis); 

• and none of the covariate controls can be affected by participation and the 
study convincingly assesses qualitatively why the instrument only affects the 
outcome via participation; 61 . 

• and, for cluster assignment, authors particularly control for external duster- 
level factors that might confound the impact of the programme (e.g. 
weather, infrastructure, community fixed effects etc.) through multivariate 
analysis. 

Score "UNCLEAR" if: 

• the exogeneity of the instrument is undear (both externally as well as why 
the variable should not enter by itself in the outcome equation) . 

• relevant confounders are controlled but appropriate statistical tests are not 
reported or exogeneity 62 of the instrument is not convincing; 

• or if insuffident details are provided on duster controls ( see category f) 
below). 

Score "NO" otherwise. 

d) For assignment- based non- randomised programme placement and self- 
selection (studies using a matching strategy or regression analysis (exduding 

IV), studies which apply other methods) 

Score "YES" if: 

• partidpants and non-partidpants are either matched based on all relevant 
characteristics explaining partidpation and outcomes; 

• or all relevant characteristics are accounted for. 63 

Score "UNCLEAR" if: 

• it is not dear whether all relevant characteristics ( only relevant time varying 
characteristics in the case of panel data regressions) are controlled. 

Score "NO" if: 

• relevant characteristics are omitted from the analysis. 


61 If the instrument is the random assignment of the treatment, the reviewer should also assess the 
quality and success of the randomisation procedure in part a) . 

62 An instrument is exogenous when it only affects the outcome of interest through affecting 
participation in the programme. Although when more than one instrument is available, statistical tests 
provide guidance on exogeneity (see background document), the assessment of exogeneity should be in 
any case done qualitatively. Indeed, complete exogeneity of the instrument is only feasible using 
randomised assignment in the context of an RCT with imperfect compliance, or an instrument 
identified in the context of a natural experiment. 

63 Accounting for and matching on all relevant characteristics is usually only feasible when the 
programme allocation rule is known and there are no errors of targeting. It is unlikely that studies not 
based on randomisation or regression discontinuity can score 'YES'' on this criterion. 
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In addition: 

dl) For non- randomised trials using panel data (including DID) models 
Score "YES" if: 

• the authors use a difference- in- differences ( or fixed effects) multivariate 
estimation method; 

• and the authors control for a comprehensive set of time- varying 
characteristics; 64 

• and the attrition rate is sufficiently low and similar in treatment and control, 
or the study assesses that drop-outs are random draws from the sample (e.g. 
by examining correlation with determinants of outcomes, in both treatment 
and comparison groups); 

• and, for cluster assignment, authors control for external duster- level factors 
that might confound the impact of the programme (e.g. weather, 
infrastructure, community fixed effects etc.) through multivariate analysis. 

Score 'UNCLEAR" if: 

• insuffident details are provided; 

• or if insuffident details are provided on duster controls. 

Score "NO" otherwise, including if the treatment effect is estimated using raw 
comparison of means in statistically un- matched groups. 

d2) For statistical matching studies including propensity scores (PSM) and 
covariate matching 65 
Score "YES" if: 

• matching is either on baseline characteristics or time- invariant 
characteristics which cannot be affeded by partidpation in the programme; 

• and the variables used to match are relevant ( e. g. demographic and 
sodoeconomic factors) to explain both partidpation and the outcome (so 
that there can be no evident differences across groups in variables that might 
explain outcomes); 

• and for PSM Rosenbaum's test suggests the results are not sensitive to the 
existence of hidden bias; 

• and, with the exception of Kernel matching, the means of the individual 
covariates are equated for treatment and comparison groups after matching; 

• and, for cluster assignment, authors control for external cluster- level factors 
that might confound the impact of the programme (e.g. weather, 
infrastructure, community fixed effects etc.) through multivariate or any 
appropriate analysis. 

Score 'UNCLEAR" if: 

• relevant variables are not included in the matching equation, or if matching 
is based on characteristics collected at endline; 

• or if insuffident details are provided on duster controls. 

Score "NO" otherwise. 

d3) For regression- based studies using cross-sectional data (exduding IV) 

Score "YES" if: 


64 Knowing allocation rules for the programme - or even whether the non- participants were individuals 
that refused to participate in the programme, as opposed to individuals that were not given the 
opportunity to participate in the programme - can help in the assessment of whether the covariates 
accounted for in the regression capture all the relevant characteristics that explain differences between 
treatment and comparison. 

65 Matching strategies are sometimes complemented with difference- in- difference regression 
estimation methods. This combination approach is usually superior since it only uses in the estimation 
the common support region of the sample size, reducing the likelihood of existence of time- variant 
unobservables differences across groups affecting outcome of interest and removing biases arising from 
time- invariant unobservable characteristics. 
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• the study controls for relevant confounders that may be correlated with both 
participation and explain outcomes (e.g. demographic and socioeconomic 
factors at individual and community level) using multivariate methods with 
appropriate proxies for unobservable covariates; 

• and a Hausman test 66 with an appropriate instrument suggests there is no 
evidence of endogeneity; 

• and none of the covariate controls can be affected by participation; 

• and, either only those observations in the region of common support for 
participants and non- participants in terms of covariates are used, or the 
distributions of covariates are balanced for the entire sample population 
across groups; 

• and, for cluster assignment, authors control particularly for external cluster- 
level factors that might confound the impact of the programme (e.g. weather, 
infrastructure, community fixed effects etc.) through multivariate analysis. 

Score "UNCLEAR" if: 

• relevant confounders are controlled but appropriate proxy variables or 
statistical tests are not reported; 

• or if insufficient details are provided on cluster controls. 

Score "NO" otherwise. 

d4) For study designs which do not account for differences between groups using 

statistical methods, score "NO". 

2) Spillovers: was the study adequately protected against performance 

bias? 


Score "YES" if: 

• the intervention is unlikely to spill over to comparisons (e.g. participants and 
non- participants are geographically and/ or socially separated from one 
another and general equilibrium effects are u nlik ely). 67 

Score "UNCLEAR" if: 

• spillovers are not addressed dearly. 

Score "NO" if: 

• allocation was at individual or household level and there are likely spillovers 
within households and communities which are not controlled for in the 
analysis; 

• or if allocation at duster level and there are likely spillovers to comparison 
dusters. 

3) Selective reporting: was the study free from outcome and analysis 

reporting biases? 

Score "YES" if: 

• there is no evidence that outcomes were selectively reported (e.g. all relevant 
outcomes in the methods section are reported in the results section); 

• and authors use "common' methods" 68 of estimation and the study does not 
suggest the existence of biased exploratory research methods. 69 


66 The Hausman test explores endogeneity in the framework of regression by comparing whether the 
ordinary least squares (OLS) and the IV approaches yield significantly different estimations. However, 
it plays a different role in the different methods of analysis. While in the OLS regression framework the 
Hausman test mainly explores endogeneity and therefore is related to the validity of the method, in IV 
approaches it explores whether the author has chosen the best available strategy for addressing causal 
attribution (since in the absence of endogeneity OLS yields more precise estimators) and therefore is 
more related to analysis reporting bias. 

67 Contamination, that is differential receipt of other interventions affecting outcome of interest in the 
control or comparison group, is potentially an important threat to the correct interpretation of study 
results and should be addressed via PICO and study coding. 
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Score "NO" if: 

• some important outcomes are subsequently omitted from the results or the 
significance and magnitude of important outcomes was not assessed; 

• or authors use uncommon or less rigorous estimation methods such as 
failure to conduct multivariate analysis for outcomes equations where it is 
has not been established that covariates are balanced. 70 

Score "UNCLEAR" otherwise. 

4) Other: was the study free from other sources of bias? 

Important additional sources of bias may include: concerns about blinding of 
outcome assessors or data analysts; concerns about courtesy bias from outcomes 
collected through self-reporting; concerns about coherence of results, for 
example between descriptive statistics and outcome questions; data on the 
baseline collected retrospectively; information is collected using an 
inappropriate instrument (or a different instrument/ at different time/ after 
different follow-up period in the comparison and treatment groups) . 

Score "YES" if: 

• the reported results do not suggest any other sources of bias. 

Score "UNCLEAR" if: 

• other important threats to validity may be present. 

Score 'NO" if: 

• it is clear that these threats to validity are present and not controlled for. 


Review question (2): Quality appraisal of studies examining barriers 
and enablers 

We assessed the quality of included studies using an adapted version of the Critical 
Appraisal Skills Programme checklist (CASP, 2006), makingjudgments on the 
adequacy of reporting, data collection, presentation, analysis and conclusions 
drawn. The checklist is presented below. In accordance with our inclusion criteria 
we filtered out studies of particularly low quality (Hannes, 20 11) and studies where 
questions 1- 5 were assessed as 'No" were excluded at this stage. The remaining 
studies were classified as of high or low quality. The results of the quality appraisal 
are reported in the review. 


68 "Common methods" refers to the use of the most credible method of analysis to address attribution 
given the data available. 

69 A comprehensive assessment of the existence of "data mining" is not feasible particularly in quasi- 
experimental designs where most studies do not have protocols and replication seems the only possible 
mechanism to examine rigorously the existence of data mining. 

70 i) For PSM and covariate matching, score "YES" if: where over 10% of participants fail to be matched, 
sensitivity analysis is used to re- estimate results using different matching methods ( Kernel Matching 
techniques). For matching with replacement, no single observation in the control group is matched 
with a large number of observations in the treatment group. Where not reported, score "UNCLEAR”. 
Otherwise, score "NO”, ii) For IV (including Heckman) models, score 'YES" if: the authors test and 
report the results of a Hausman test for exogeneity (p<o.os is required to reject the null hypothesis of 
exogeneity), the coefficient of the selectivity correction term (Rho) is significantly different from zero 
(P<0.05) (Heckman approach). Where not reported, score "UNCLEAR”. Otherwise, score "NO”, iii) For 
studies using multivariate regression analysis, score 'YES” if: authors conduct appropriate specification 
tests (e.g. reporting results of multicollinearity test, testing robustness of results to the inclusion of 
additional variables etc.). Where not reported or not convincing, score 'UNCLEAR”. Otherwise, score 
"NO". 
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Critical appraisal of qualitative studies included to answer review question (2) 


No 

Question 

Id 

Code 

1 

Is the research aim dearly stated? 

Research aim 

Yes=l 

No=0 

Partially=8 

2 

Description of the context? 

Context 

Yes=l 

No=0 

Partially=8 

3a 

Description of sampling procedures 
- How have the partidpants been seieded, were they the 
most appropriate? 

Sampling 

Yes=l 

No=0 

Partially=8 

Unclear=9 

3b 

Please describe sampling procedures 

Sampling description 

Free answer 

4 

Are sample characteristics suffidently reported? (sample 
size, location, and at least one additional charaderistic) 

Sampling characteristics 

Yes=l 

No=0 

Partially=8 

Undear=9 

5a 

Is it dear how the data were colleded; e.g. for interviews, 
is there an indication of how interviews were conducted? 

Data collection 

Yes=l 

No=0 

Partially=8 

Undear=9 

5b 

Please describe data collection methods 

Data collection description 

Free answer 

6a 

Methods of recording of data reported? 

Data recording 

Yes=l 

No=0 

Partially=8 

Undear=9 

6b 

Please describe methods recording data 

Data recording description 

Free answer 

7a 

Methods of analysis explidtly stated? 

Analysis 

Yes=l 

No=0 

Partially=8 

Undear=9 

7b 

Please describe methods of analysis 

Analysis description 

Free answer 

8 

Is there a clear link to relevant literature/ theoretical 
framework? 

Link to relevant 
literature/ theory 

Yes=l 

No=0 

Partially=8 

9a 

Is the design appropriate to answer the research 
question? 

- Has the researcher justified the research design? 

Appropriate methodology 

Yes=l 

No=0 

Partially=8 

Undear=9 

9b 

If answer to Q9a is no or partially, justify 

Methodology comment 

Free answer 

10a 

Was the sampling strategy appropriate to the aims of the 
research? 

- Have the researchers explained how the partidpants 
were selected? 

- Have the researchers explained why the partidpants 
they selected were the most appropriate to provide access 
to the type of knowledge sought by the study? 

- Have the researchers discussed issues around 
recruitment? (e.g. why some people chose not to take 
part) 

Appropriate sampling 

Yes=l 

No=0 

Partially=8 

Unclear=9 

10b 

If answer to QlOa is no or partially, justify 

Sampling comment 

Free answer 
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11a Were the data collected in a way that addressed the 
research issue? 

- Were the methods used appropriate and justified? 

- Did the researcher discuss saturation of data? 


Appropriate methods of data 
collection 


Yes=l 

No=0 

Partially=8 

Undear=9 


lib If answer to Qlla is no or partially, justify Data collection comment Free answer 

12a Was the data analysis sufficiently rigorous? Appropriate analysis Yes=l 

- Is there a detailed description of the analysis process? No=0 

- Do the data support the findings? Partially=8 

- Is the relationship between the researcher and the Undear=9 


partidpants adequately considered? 

- To what extent are contradictory data taken into 
account? 

- If the findings are based on quantitative analysis of 
survey data, are multivariate techniques used to control 
for potential confounding variables? 


12b If answer to Q12 is no, partially or undear, justify 

Analysis comment 

Free answer 

13a Has triangulation been applied? 

- Data triangulation (location, time and partidpants) 

- Investigator triangulation 

- Theory triangulation (several theories) 

- Methodological triangulation 

Triangulation 

Yes=l 

No=0 

Partially=8 


13b Describe triangulation methods employed Triangulation description 

14a Are the analysis and condusions clearly presented? Clarity of analysis and Yes=l 

- Have the researchers discussed the credibility of their condusions No=0 

findings? (e.g. triangulation, respondent validation, more Partially=8 

than one analyst) Undear=9 

- Is there adequate discussion of the evidence both for 
and against the researcher's arguments? 

- Are the findings explidt? 

- Are the findings discussed in relation to the original 
research question? 


14b 

If answer to Q14a is no or partially, justify 

Condusions comment 

Free answer 

15 

Was the potential for conflict of interest considered and 
addressed? 

Conflid of interest considered 
and addressed 

Yes=l 

No=0 

Partially=8 

Unclear=9 

16 

Does the paper discuss ethical considerations related to 
the research? 

Ethical considerations 

Yes=l 

No=0 

Partially=8 

17 

Comments/justification about overall reporting and 
methdology 

Comments/justification 

Free answer 
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Appendix D 


EFFECT SIZE CALCULATIONS 


For studies using a parallel group or statistical matching- based strategy, the 
response ratio (RR) and its standard error (SErr) were estimated as follows 
(Borenstein et al., 2009): 


where F t is the mean outcome in the treatment group, F c is the mean outcome in 
the comparison group, n t and m are the sample sizes of the treatment and 
comparison groups respectively, Sp is the pooled standard deviation and Sc and St are 
the standard deviations in treatment and comparison. 71 For regression- based 
studies, RR and its standard error were estimated as follows (Borenstein et al., 
2009): 72 


where /? is the coefficient of the treatment variable in the regression and a the pooled 
standard deviation. 73 Often Sp or a were not reported, therefore we calculated SErr 
by rescaling RR using information reported on statistical significance such as a t- 
statistic or p- value associated with the ft coefficient: e.g. ln(SERR) =ln(RR)/ty. 

Borenstein et al. (2009) indicate response ratios should only be used when the 
outcome is measured on a true ratio scale and has a natural "zero" point. Thus, the 
RR is not meaningful in some cases of outcomes measured in FFS studies, such as 
knowledge test scores and indices of adoption not related to pesticide use. In such 
cases, we have therefore calculated the Hedges g (sample size corrected) 
standardised mean difference. For studies using parallel group or matching 
strategies g and its standard error were computed using the previous notation as 
follows (Borenstein et al., 2009): 


71 Note that for studies using a matching strategy the outcome level for the treatment group and control 
group used to estimate the effect size is the outcome level for each group after matching. If kernel 
matching is used substitute Y c withF r ATET (average treatment effect on the treated). 

72 For some maximum likelihood regression models such as Logit or Probit (for dichotomous 
outcomes) and Tobit (for continuous outcomes), it is not possible to use the regression coefficient to 
estimate the RR. In such a case, p refers to the "impact effect” calculated from the regression coefficient 
for Logit, Probit or Tobit models. For semi- logarithmic difference- in- differences multivariate 
regression we calculate the response ratio asRR = e !i * 100 . 

73 There are two main approaches to the calculation of the pooled standard deviation from regression- 
based studies: in Cohens approach 8 is the standard deviation of the dependent variable across all 
observations; in Hedges approach 8 is the standard deviation of the error term in the regression. 



(rc c -l)*Sg + (n t -l)*Sj 
n t +n c - 2 
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For studies using regression analysis g and its standard error were estimated 
following Keef and Roberts (2004): 74 


g = c(v) SE g = — * (-j + v * [c(v )] 2 — 17 + 2 ) , where 


1 _ Jv 

C(v) yj 2 




where [3 refers to the coefficient of the treatment variable in the regression, d is the 
standard deviation of the regression residuals, v is n-k degrees of freedom and r() is 
the gamma function. We also calculated SE g by rescaling g p using information 

reported on statistical significance of /?, such as a t-statistic or p- value: e.g. SE a - — 


In all cases where g was calculable, we were able to calculate RR. In cases where we 
were not able to calculate g but were able to calculate RR, we translated RR into g 
using the Cox transformation gcox=ln(OR)/ 1.65 in order to minimise loss of 
information ( Sanchez- Meca et al., 2003). Recognising that this method is applicable 
to the log- odds ratio, we conducted sensitivity analysis according to whether the 
magnitude of g is systematically affected by this transformation. 


SYNTHETIC EFFECT SIZES 


Where multiple dependent effect sizes were presented in a single study for a 
particular outcome construct, we calculated synthetic effect size point estimate using 
the sample- weighted average, and applying appropriate formulae to recalculate 
variances according to Borenstein et al. (2009, chapter 24): 

( m \ m 

J = ^Vi+^Jrij^Vj) 

i = 1 7 i = 1 i*j 

We calculated the correlation between estimates as the sample- weighted mean of 
the correlation of treatment groups and the correlation of the comparison groups. 
The correlation between control arms was assumed 1 where the same control group 
is used as comparator, and 0 otherwise. The correlation between treatment arms 
was assumed to be 0 when combining results from different treatment groups, and 1 
when combining results from the same treatment groups over time. When 
combining results across different individuals within the same treatment group the 
correlation was assumed to be 0.5, which estimates variance at the mid- point 
between the two extreme cases of treating comparisons as independent (with 
correlation coefficient equal to 0, and likely underestimating the variance), or 
treating them as perfectly correlated (correlation coefficient of 1, and likely 
overestimating the variance) . 


META-ANALYSIS OF BIVARIATE AND PARTIAL EFFECT 
SIZES 


According to Keef and Roberts (2004, p. 103), effect sizes are only strictly 
comparable in studies employing a common model, meaning that "suitable proxies 
for the same constructs [i.e. the outcome variable, treatment variable and 
covariates] are included in all the studies being synthesized", and the same 
estimation methods are used to estimate the treatment effect (Becker &Wu, 2007). 
There are several obvious reasons for such differences. Firstly, the coefficient may be 


74 For studies with large n, c(v) is considered equal to 1. 
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biased due to model specification errors, for example due to omitted variables bias 
(confounding) or multicollinearity. Multicolfinearity is particularly problematic for g 
since it inflates the standard error of the regression a, thus reducing the absolute 
value of g (Keef & Roberts, 2004). In addition, while multicollinearity does not 
cause bias in the predicted values of the dependent variable y , it may cause the 
correlated coefficient estimates to be biased individually in misspedfied models. 

However, even where there are no biases arising from specification error, it is 
important to note that the partial /? coefficient is measuring the treatment effect 
conditional on the covariates, and is therefore measuring a different quantity to the 
bivariate relationship, due to collinearity. In the case of the treatment effect 
regression, this would be represented as significant interaction effects between 
treatment variable and correlated covariates, thus violating the "homogeneity of 
slopes" assumption (the slope coefficients for the covariates are not homogeneous 
between treatment and comparison group) (Keef & Roberts, 2004). Where 
treatment variable X and covariates Z are uncorrelated, the adjusted value of /? a is 
equal to the unadjusted bivariate value /? u (equal to the sum of the partial correlation 
between X and outcome variable Y, p XY , and ratio of standard deviation of Y to 
standard deviation of X) : 

Pa — Pu — Pyx 

°x 

However, where X and Z are correlated by p xz , the adjusted value of /? is: 


r -. 


r 1 pyzPxz- 

Pyx~PyzPxz °Y 

ii 

2 * 

pyx 

1_ Pxz °x_ 

1_ Pxz 


which may be smaller or larger than p u depending on the relative magnitude of the 
partial correlations between Y and X, and X and Z. 

Determining the extent that multicollinearity is problematic can potentially be 
operationalised through risk of bias assessment, including through assessment of 
risk of bias due to specification error (e.g. whether primary authors use theoretically 
derived "reduced form" models) and multicollinearity (e.g. use of interaction terms 
and reporting of statistical diagnostic tests) . 

A review of the literature identified several other solutions to the problem: 

1) Becker and Wu (2007) propose a generalised least squares meta-analysis 
approach which utilises the variance- covariance matrices of the primary 
study regression models in the meta- analysis weighting. 

2) Adopting ad hoc assumptions regarding correlations among covariates, 
Becker and Wu (2007) state that reviewers could impose a common 
correlation among slopes (giving an example correlation of 0.2) and 
estimating generalised least squares meta- analysis. 

3) Others propose synthesising bivariate and partial effect sizes separately and 
coding information on the presence or absence of variables included in the 
multivariate specifications which generated the partial effect sizes, and 
exploring whether these factors are systematically correlated with effect size 
estimates in sensitivity analysis (Aloe & Becker, 2012). 

4 ) Transforming partial effect sizes to approximate the bivariate effect size, as 
proposed by Peterson and Brown (2005) for the correlation coefficient. 
However, this method is not recommended by Aloe and Thompson (2013) 
due to uncertainties about the properties of the resulting effect size point 
estimate and variance. 
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5) Synthesising the semi- partial correlation coefficient effect size r sp in place of 


the standardised slope coefficient (Aloe & Thompson, 20 13) : r sp = 


t^l-R 


yjn—p—l ' 


where Ry is the squared multiple correlation for the regression model (equal 
to the coefficient of determination for a linear model including constant), tp 


is the t- statistic associated with the treatment effect coefficient /? and n-p- 1 
the degrees of freedom. However, the drawback of the approach is the 
correlation coefficient does not represent the magnitude of the change in 
outcomes and is therefore of less relevance for policy. 
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Appendix E 


INCLUDED EFFECTIVENESS STUDY DESCRIPTIVES 


Table El Included impact evaluations by region: study descriptive information 


Study name 

Intervention 

type 

Country: programme 
name 

Crop 

Study 

design 

Overall 
risk of 
bias 

Outcome data reported for FFS participants (Code: X effect size calculable; x 
outcome measured but effect size not calculable; * standard errors calculable) 

Cost 

data 






Knowledge 

Adoption 

Agriculture 

outcome 

Health Environment Empowerment 
outcome outcome outcome 


East Asia and the Pacific 

Chi et al., 1999 

IPM-FFS 

Vietnam: FAO National I PM 
Programme, Thoi Long 

Rice 

Cross-section 

High risk 
of bias 

X 

X 




Feder et al., 2004 

IPM-FFS 

Indonesia: National I PM 
Training Project Phase II 
(FAO, World Bank, Gol) 

Rice 

Lonqitudinal 

(DID 

regression) 

Medium 
risk of bias 


X* 

X* 



Haiyang, 2002 

IPM-FFS 

China: Netherlands Poverty 
Alleviation Project (CNPAP), 
Huoshan 

Rice 

Cross-section 

High risk 
of bias 


X 

X 

X 

X 

Huan et al., 1999 

IPM-FFS 

Vietnam: FAO Programme for 

Rice 

Cross-section 

High risk 

X* 

X* 

X* 




Community I PM in Asia of bias 
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Study name 

Intervention 

type 

Country: programme 
name 

Crop 

Study 

design 

Overall 
risk of 
bias 

Outcome data reported for FFS participants (Code: X effect size calculable; x 
outcome measured but effect size not calculable; * standard errors calculable) 

Cost 

data 






Knowledge Adoption 

Agriculture 

outcome 

Health Environment Empowerment 
outcome outcome outcome 


Kabir&Uphoff, 

2007 

ICM-FFS 

Myanmar: Farmer field school 
for sustainable agriculture 
development 

Rice 

Cross-section 

High risk 
of bias 


X 



Mangan &Mangan, 
1998 

IPM-FFS 

China: IPM-FFS Sichuan 
Province (FAO) 

Rice 

Longitudinal 

High risk 
of bias 




X 

Mariyono, 2007 

IPM-FFS 

Indonesia: National I PM 
Training Project Phase II 
(FAO, World Bank, Gol) 

Rice 

Longitudinal 

High risk 
of bias 

X 




Murphy et al., 2002 

IPM-FFS 

Vietnam: FAO National I PM 
Program, Nam Dinh 

Potato, 

vegetable 

Cross-section 

High risk 
of bias 

X* 




Norvell &Hammig, 
1999 

IPM-FFS 

Indonesia: National I PM 
Training Project Phase II 
(FAO, World Bank, Gol) 

Vegetable 

Cross-section 

(regression) 

High risk 
of bias 

X* 




Ooi and Kenmore, 
2005 

IPM-FFS 

China, Vietnam: FAO IPM-FFS 
programmes for East & South 
Asia 

Cotton 

Longitudinal 

High risk 
of bias 


X 



Palis, 1998 

IPM-FFS 

+input 

+marketing 

support 

Philippines: Barangay 
Integrated Pest Management 
(BIPM) project. Central Luzon 
(IRRI, FAO, Philrice) 

Rice 

Longitudinal 

High risk 
of bias 

X 

X* 



Pananurak, 2010 

IPM-FFS 

China: FAO/ EU IPM 
Programme for Cotton in Asia 

Cotton 

Longitudinal 

(DID 

regression) 

Medium 
risk of bias 

X* 

X* 
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Study name 

Intervention 

type 

Country: programme 
name 

Crop 

Study 

design 

Overall 
risk of 
bias 

Outcome data reported for FFS participants (Code: X effect size calculable; x Cost 

outcome measured but effect size not calculable; * standard errors calculable) data 






Knowledge 

Adoption Agriculture Health Environment Empowerment 
outcome outcome outcome outcome 

Praneetvatakul & 
Waibel, 2006 

IPM-FFS 

Thailand: FAO/EUIPM 
Programme for Cotton in Asia 

Rice 

Longitudinal 

(DID 

regression) 

Medium 
risk of bias 


X* X* 

Price, 2001 

IPM-FFS 

Philippines: Central Luzon 
(IRRI, FAO, Philrice) 

Rice 

Cross-section 

High risk 
of bias 

X* 

X 

Rejesusetal., 2010 

IPM-FFS 

Vietnam: FAO Programme for 
Community I PM in Asia 

Rice 

Longitudinal 

(DID- 

regression) 

Medium 
risk of bias 

X* 

X* X* 

Rolaetal., 2002 

IPM-FFS 

Philippines: FAO National I PM 
Programme 

Rice 

Cross-section 

High risk 
of bias 

X 


Van de Fliert, 2000 

ICM-FFS 

Indonesia: National I PM 
Training Project Phase II 
(FAO, World Bank, Gol) 

Potato 

Cross-section 

(regression) 

High risk 
of bias 


X X 

Walter- Echols & 
Soomro, 2005 

IPM-FFS 

China: FAO/ EU IPM 
Programme for Cotton in Asia 

Cotton 

Longitudinal 

High risk 
of bias 


X 

Walter- Echols & 
Soomro, 2005 

IPM-FFS 

Vietnam: FAO/EUIPM 
Programme for Cotton in Asia 

Cotton 

Longitudinal 

High risk 
of bias 


X 

Wu, 2010 

IPM-FFS 

China: FAO/EUIPM 
Programme for Cotton in Asia 

Cotton 

Lonqitudinal 

(DID 

regression) 

Medium 
risk of bias 


X* X* 

Yamazaki & 
Resosudaimo, 2008 

IPM-FFS 

Indonesia: National IPM 
Training Project Phase II 
(FAO, World Bank, Gol) 

Rice 

Longitudinal 

(DID 

regression) 

Medium 
risk of bias 


X* X* 

Yangetal., 2005 

IPM-FFS 

China: FAO/ EU IPM 
Programme for Cotton in Asia, 

Cotton 

Cross-section 

High risk 
of bias 


X* X* 
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Study name 


Intervention Country: programme Crop Study 

type name design 


Overall Outcome data reported for FFS participants (Code: X effect size calculable; x Cost 

risk of outcome measured but effect size not calculable; * standard errors calculable) data 

bias Knowledge Adoption Agriculture Health Environment Empowerment 

outcome outcome outcome outcome 


Hubei Province 


Yangetal., 2008 

IPM-FFS 

China: FAO vegetable I PM, 
Yunnan Province 

Vegetable 

Longitudinal 

(DID 

regression) 

High risk 
of bias 





Yorobeetal., 2011 

IPM-FFS 

Philippines: IPM Collaborative 
Research Support Program 
(CRSP), Nueva Edja (USAID) 

Vegetable 

Cross-section 

(IV 

regression) 

Medium 
risk of bias 


X* 



Latin America 

Bentley et al., 2007 

IPM-FFS 

Bolivia: potato integrated 
management (CIP-PROINPA) 

Potato 

Cross-section 

High risk 
of bias 




X 

Cavatassi et al., 2011 

IPM-FFS 

+input 

+marketing 

support 

Ecuador: Plataformas Program 
(FAO) 

Potato 

Cross-section 

(PSM-WLS 

regression) 

Medium 
risk of bias 


X* 

X* 


Coleetal., 2007 

IPM-FFS 

+input 

+marketing 

support 

Ecuador: EcoSalud 

Potato 

Cross-section 

High risk 
of bias 


X* 



Godtland et al., 
2004 

IPPM-FFS 

Peru: CIP CARE Andean FFS 

Vegetable 

Cross-section 

(PSM) 

Medium 
risk of bias 

X* 




Labarta, 2005 

IPM-FFS 

Nicaragua: Project for I PM in 
Central America (PROMIPAC) 

vegetable 

Cross-section 

(IV 

regression) 

High risk 
of bias 


X* 

X* 

X* 

Mauceri et al., 2007 

IPM-FFS 

Ecuador: IPM Collaborative 
Research Support project 

Potato 

Cross-section 

(IV 

High risk 
of bias 


X* 


X 


(CRSP), Carchi regression) 
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Study name 

Intervention 

type 

Country: programme 
name 

Crop 

Study 

design 

Overall 
risk of 
bias 

Outcome data reported for FFS participants (Code: X effect size calculable; x 
outcome measured but effect size not calculable; * standard errors calculable) 

Cost 

data 






Knowledge Adoption Agriculture 

outcome 

Health Environment Empowerment 
outcome outcome outcome 


Orozco Qrilo et al., 
2008 

"Integrated 

management"- 

FFS 

Mexico: El Proyecto Manejo 
Sustentable de Laderas 
(PMSL) 

Cereal 

Longitudinal 

High risk 
of bias 

X* X* X* 



Rebaudo & Dangles, 
2011 

IPM-FFS 

Ecuador undear potato IPM 
(CIP) 

Potato 

Cross-section 

High risk 
of bias 

X* 



Torrez et al., 1999 

IPM-FFS 

Bolivia: potato integrated 
management (CIP-PROINPA) 

Potato 

Cross-section 

High risk 
of bias 

X* X* 



Van Rijn, 2010 

ICM-FFS 

+input 

+marketing 

support 

Peru: DE Foundation Coffee 
Project 

Coffee 

Cross-section 

(PSM) 

Medium 
risk of bias 

X* X* 

X* 


Zuger, 2004 

IPM/ ICM-FFS 

Peru: Cajamarca FFS (CARE, 
CIP) 

Potato 

Cross-section 

(regression) 

High risk 
of bias 

x x X* 


X 

Central Asia 

Dinpanah et al., 
2010 

FFS (unclear) 

Iran: unspecified FFS 
programmes 

Rice 

Cross-section 

High risk 
of bias 

X* X* X* 


X 

South Asia 

Ali & Sharif, 2012 

IPM-FFS 

Pakistan: Cotton IPM 
programme (FAO, EU, GoP) 

Cotton 

Cross-section 

(PSM) 

Medium 
risk of bias 

X* X* X* 



Birthal et al., 2000 

IPM-FFS 

+input 

support 

India: Maharashtra project 
(National Centre for Integrated 
Pest Management) 

Cotton 

Cross-section 

High risk 
of bias 

X* X* 


X 

DAM DA, 2011 

ICM-FFS 

+input 

+marketing 

Bangladesh: Agriculture Sector 
Programme Support 2 
(DAMDA, MoA) 

Rice 

Cross-section 

(PSM) 

High risk 
of bias 

X 

X 

X 


support 
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Study name 

Intervention 

type 

Country: programme 
name 

Crop 

Study 

design 

Overall 
risk of 
bias 

Outcome data reported for FFS participants (Code: X effect size calculable; x 
outcome measured but effect size not calculable; * standard errors calculable) 

Cost 

data 






Knowledge 

Adoption Agriculture 
outcome 

Health Environment Empowerment 
outcome outcome outcome 


Islam et al., 2006 

IPM-FFS 

Bangladesh: Strengthening 
Plant Protection Services 
(SPPS) Project 

Rice 

Cross-section 

High risk 
of bias 

X 

X X 


X 

J ones, 2002 

ICM-FFS 

Sri Lanka: INTEGRATED IPM 
project (CARE) 

Rice 

Cross-section 

High risk 
of bias 


X X 



KhanetaL, 2007 

IPM-FFS 

Pakistan: National IPM 
Programme, Khairpur 

Cotton 

Longitudinal 

High risk 
of bias 

X* 

X* X* 



LamaetaL, 2003 

IDM-FFS 

Nepal: CIPIDMFFS 

Potato 

Cross-section 

High risk 
of bias 


X 



Mandni & J iggins, 
2008 

IPM-FFS 

India: FAO/EU IPM 
Programme for Cotton in Asia 

Cotton 

Longitudinal 

High risk 
of bias 

X 

X* X* 



Naiketal., 2010 

ICM-FFS 

India: Karnataka Community 
Based Tank Management 
Proj ect ( KCBTMP) 

Cereal, 

groundnut 

Cross-section 

High risk 
of bias 


X* 



Ooi and Kenmore, 
2005 

IPM-FFS 

Bangladesh, India, Pakistan: 
various FAO IPM-FFS 
programmes 

Cotton 

Longitudinal 

High risk 
of bias 


X X 



Pananurak, 2010 

IPM-FFS 

India: FAO/EU IPM 
Programme for Cotton in Asia 

Cotton 

Lonqitudinal 

(DID 

regression) 

High risk 
of bias 


X* X* 

X* 


Pananurak, 2010 

IPM-FFS 

Pakistan: FAO/EU IPM 
Programme for Cotton in Asia 

Cotton 

Lonqitudinal 

(DID 

regression) 

Medium 
risk of bias 


X* X* 

X* 


Pandeetal., 2009 

ICM-FFS 

Nepal: ICMFFS 

Rice 

Cross-section 

High risk 


X* 


X 


+input of bias 

+marketing 

support 
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Study name 

Intervention 

type 

Country: programme 
name 

Crop 

Study 

design 

Overall 
risk of 
bias 

Outcome data reported for FFS participants (Code: X effect size calculable; x 
outcome measured but effect size not calculable; * standard errors calculable) 

Cost 

data 






Knowledge 

Adoption 

Agriculture 

outcome 

Health Environment Empowerment 
outcome outcome outcome 


Pouchepparadjou et 
al., 2005 

IPM-FFS 

India: DoA Rice I PM project, 
Pondicherry 

Rice 

Cross-section 

(regression) 

High risk 
of bias 






Raoetal., 2012 

ISNM-FFS 

India: Sree Ram Sagar Project 
(SRSP), Andhra Pradesh 

Rice 

Cross-section 

High risk 
of bias 

X* 

X* 




Reddy & Suiyamani, 
2005 

IPM-FFS 

India: FAO/EUIPM 
Programme for Cotton in Asia 

Cotton 

Longitudinal 

High risk 
of bias 

X* 





Ricker- Gilbert et al., 
2008 

IPM-FFS 

Bangladesh: National I PM 
Programme (FAO) 

Rice, 

vegetable 

Cross-section 

(IV 

regression) 

High risk 
of bias 


X* 



X 

Tripp etal., 2005 

IPM-FFS 

Sri Lanka: FFS for I PM in rice 
Southern Province (DoA, FAO) 

Rice 

Cross-section 

High risk 
of bias 

X 

X 




Van den Berg and 
Amarasinghe, 2003 

IPM-FFS 

Sri Lanka: IPVM Project (FAO, 
UNEP) 

Rice 

Cross-section 

High risk 
of bias 


X* 

X* 


X 

Walter- Echols & 
Soomro, 2005 

IPM-FFS 

Bangladesh: FAO/EU IPM 
Programme for Cotton in Asia 

Cotton 

Longitudinal 

High risk 
of bias 


X 




Walter- Echols & 
Soomro, 2005 

IPM-FFS 

India: FAO/EUIPM 
Programme for Cotton in Asia 

Cotton 

Longitudinal 

High risk 
of bias 


X 


X 


Walter- Echols & 
Soomro, 2005 

IPM-FFS 

Pakistan: FAO/EU IPM 
Programme for Cotton in Asia 

Cotton 

Longitudinal 

High risk 
of bias 


X 


X 


Williamson et al., 
2003 

IPM-FFS 

India: Insecticide Resistance 
Management (IRM), 
Maharashtra 

Cotton 

Cross-section 

High risk 
of bias 


X 

X 
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Study name 

Intervention 

type 

Country: programme 
name 

Crop 

Study 

design 

Overall 
risk of 
bias 

Outcome data reported for FFS participants (Code: X effect size calculable; x Cost 

outcome measured but effect size not calculable; * standard errors calculable) data 






Knowledge 

Adoption 

Agriculture 

outcome 

Health Environment Empowerment 
outcome outcome outcome 

Sub-Saharan Africa 

Achonga et al., 2011 

"Agrobio- 

diversity"-FFS 

Kenya: Bondo Food Security 
Project (FAO) 

Diverse 

crops 

Cross-section 

(univariate 

regression) 

High risk 
of bias 





Amera, 2009 

IPM-FFS 

Kenya: Cotton I PM project 
(EU& PAN-UK) 

Cotton 

Cross-section 

High risk 
of bias 


X* 


X* 

Bunyatta et al., 2006 

ISM-FFS 

Kenya: KARI Farmer Field 
School Project 

Potato, 

vegetable 

Cross-section 

High risk 
of bias 

X* 

X* 



Carlberg et al., 20 12 

IPM-FFS 

Ghana: Peanut Collaboration 
Research Support Program 
(CRSP) 

Groundnut 

Cross-section 

(IV 

regression) 

High risk 
of bias 



X* 


Dankyi etal., 2005 

IPPM-FFS 

Ghana: Peanut Collaboration 
Research Support Program 
(CRSP) 

Groundnut 

Cross-section 

High risk 
of bias 

X 

X 



David, 2007 

ICPM-FFS 

Cameroon: Sustainable Tree 
Crops Program (STCP) Phase 
II 

Cocoa 

Cross-section 

High risk 
of bias 

X* 

X 



David & Asamoah, 
2011 

IPPM-FFS 

Ghana: National Cocoa Disease 
and Pest Control Program 
(CODAPEC) 

Cocoa 

Cross-section 

(regression) 

High risk 
of bias 

X* 




Davis etal., 2012 

IPPM-FFS 

Kenya: East African Sub- 
regional Pilot Project Phase II 
(FAO) 

Vegetable 

Longitudinal 

(covariate 

matching) 

Medium 
risk of bias 



X* 


Davis etal., 2012 

IPPM-FFS 

Tanzania: East African Sub- 
regional Pilot Project Phase II 
(FAO) 

Vegetable 

Longitudinal 

(covariate 

matching) 

Medium 
risk of bias 



X* 


Davis etal., 2012 

IPPM-FFS 

Uganda: East African Sub- 
regional Pilot Project Phase II 

Vegetable 

Longitudinal 

(covariate 

High risk 
of bias 



X* 



(FAO) matching) 
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Study name 

Intervention 

type 

Country: programme 
name 

Crop 

Study 

design 

Overall 
risk of 
bias 

Outcome data reported for FFS participants (Code: X effect size calculable; x Cost 

outcome measured but effect size not calculable; * standard errors calculable) data 






Knowledge 

Adoption 

Agriculture 

outcome 

Health Environment Empowerment 
outcome outcome outcome 

De J ager et al., 2009 

ISM-FFS 

+marketing 

support 

Kenya: INM Pilot Project in 
Central and Eastern Kenya 

Cereals, 

vegetable 

Cross-section 

High risk 
of bias 

X 

X 



Endalew, 2009 

IPM-FFS 

Ethiopia: J imma and Sidama 
FFS 

Coffee 

Cross-section 

High risk 
of bias 

X* 

X* 



Erbaugh et al., 2010 

IPM-FFS 

Uganda: I PM Collaborative 
Research Support Program 
(CRSP) 

Cereals, 

vegetable, 

groundnut 

Cross-section 

High risk 
of bias 

X* 




Friis- Hansen & 
Duveskog, 2012 

IPPM-FFS 

Kenya: East African Sub- 
regional Pilot Project Phase II 
(FAO) 

Vegetable 

Cross-section 

High risk 
of bias 




X 

Friis- Hansen & 
Duveskog, 2012 

IPPM-FFS 

Tanzania: East African Sub- 
regional Pilot Project Phase II 
(FAO) 

Vegetable 

Cross-section 

High risk 
of bias 




X 

Friis- Hansen & 
Duveskog, 2012 

IPPM-FFS 

Uganda: East African Sub- 
regional Pilot Project Phase II 
(FAO) 

Vegetable 

Cross-section 

High risk 
of bias 




X 

Friis- Hansen et al., 
2004 

IPPM-FFS 

+input 

+marketing 

support 

Uganda: East African Sub- 
regional Pilot Project Phase II 
(FAO) 

Vegetable 

Cross-section 

High risk 
of bias 


X* 


X* 

Gockowski et al., 
2005 

ICPM-FFS 

Nigeria: Sustainable Tree 
Crops Program (STCP) Phase 
II (IITA) 

Cocoa 

Cross-section 

High risk 
of bias 


X 

X 
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Study name 

Intervention 

type 

Country: programme 
name 

Crop 

Study 

design 

Overall 
risk of 
bias 

Outcome data reported for FFS participants (Code: X effect size calculable; x Cost 

outcome measured but effect size not calculable; * standard errors calculable) data 






Knowledge 

Adoption 

Agriculture 

outcome 

Health Environment Empowerment 
outcome outcome outcome 

Gockowski et al., 
2010 

ICPM-FFS 

+input 

support 

Ghana: National Cocoa Disease 
and Pest Control Program 
(CODAPEC) 

Cocoa 

Cross-section 

(regression) 

High risk 
of bias 


X* 

X* 


Hiller etal., 2009 

IPPM-FFS 

Kenya: Kenya Tea 
Development Agency/ Lipton 
Sustainable Agriculture Project 

Tea 

Cross-section 
/ longitudinal 

High risk 
of bias 

X* 

X 

X* 

X 

Kelemework, 2005 

"Integrated 
management 
practices"- FFS 

Ethiopia: Integrated 
Management of Potato Late 
Blight (CIP, IFAD) 

Potato 

Longitudinal 

High risk 
of bias 

X* 

X* 



Khalid, n.d. 

IPM-FFS 

Sudan: Gezira Scheme, FAO 
IPM in Vegetables 

Vegetable 

Cross-section 

High risk 
of bias 


X* 



Lund etal., 2010 

IPM-FFS 

Benin: Healthy Vegetables 
Through Participatory IPM in 
Peri-Urban Gardens Project 

Vegetable 

Cross-section 

High risk 
of bias 

X 




Maumbe & Swinton, 
2003 

IPPM-FFS 

Zimbabwe: Cotton IPM-FFS, 
Sanyati district 

Cotton 

Cross-section 

(regression) 

High risk 
of bias 





Mutandwa & 
Mpangwa, 2004 

IPPM-FFS 

Zimbabwe: Cotton IPM-FFS, 
Sanyati district 

Cotton 

Cross-section 

High risk 
of bias 

X* 

X* 

X* 


Odeyemi et al., 2005 

ICM-FFS 

Nigeria: Ogun State 

Unknown 

Cross-section 

High risk 
of bias 

X 




Olanyaetal., 2010 

IDM-FFS 

Uganda: CIP IDM FFS (IFAD) 

Potato 

Cross-section 

High risk 






of bias 
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Study name 

Intervention 

type 

Country: programme 
name 

Crop 

Study 

design 

Overall 
risk of 
bias 

Outcome data reported for FFS participants (Code: X effect size calculable; x 
outcome measured but effect size not calculable; * standard errors calculable) 

Cost 

data 






Knowledge 

Adoption 

Agriculture 

outcome 

Health Environment Empowerment 
outcome outcome outcome 


Rusike et al., 2004 

IPPM-FFS 

Zimbabwe: Cotton IPM-FFS 

Cereal 

Cross-section 

High risk 
of bias 

X 

X 



X 

Todo &Takahashi, 
2011 

'Participatory 

forest 

management"- 

FFS 

+input 

+marketing 

support 

Ethiopia: Participatory Forest 
Management Project in the 
Belete-Gera Regional Forest 
Priority Area 

Vegetable 

Longitudinal 

(PSM-DID 

regression) 

Medium 
risk of bias 


X* 

X* 



Waartsetal., 2012 

"Good 
agricultural 
practices"- FFS 

Kenya: Kenya Tea 
Development Agency/ Lipton 
Sustainable Agriculture Project 

Tea 

Cross-section 

(regression) 

High risk 
of bias 

X* 

X 

X* 



Wandji etal., 2007 

IPM-FFS 

Cameroon: Sustainable Tree 
Crops Program (STCP) Phase 
II 

Cocoa 

Cross-section 

(regression) 

High risk 
of bias 



X 



Williamson et al., 
2003 

IPM-FFS 

Kenya: CABI Bioscience FFS 
pilot project 

Tomato 

Cross-section 

High risk 
of bias 


X 

X 




Notes: I PM: integrated pest management; IPPM: integrated production and pest management; ICM: integrated crop management; ICPM: integrated crop and pest management; IDM: integrated disease 
management; ISM: integrated soil management; ISNM: integrated soil nutrient management. 
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DETAILED OUTCOMES REPORTED 


Table E2 Detailed outcomes variable: Knowledge and adoption 

Study 

Knowledge variable used 

Other adoption variable used 

Ali and Sharif, 2012 

* Probability: farmer can differentiate beneficial and harmful pests 

* Probability farmer adopted positive practices (pooled variable of percentage farmers 
adopting: pest scouting, pesticide timing, recommended varieties, deep ploughing) 

Amera, 2009 


* Percentage of farmers that use IPM techniques 

Bunyatta et al., 2006 

* Test score: mean knowledge of soil and crop management technology 

* Mean score of an index measuring adoption of soil management technology 

Chi et al., 1999 

* Test score: pooled variable of mean entomological knowledge, insect- plant 
interaction knowledge and insecticide knowledge 

* Percentage of farmers that applied insect control practices (pooled variable of baits, 
water management, small ducks) 

Dankyi etal., 2005 

* Probability: percentage of farmers that have knowledge: pooled variable of 
appropriate practices (recognise foliar pests early enough to apply a control 
measure, recognise beneficial insects or natural enemies) 

* Percentage of individuals that used positive practices (pooled variable of test seeds 
before planting, plant resistance cultivars, disease control through spray, disease control 
through rogue, pest control through traps set, carried out row planting) 

David &Asamoah, 2011 

* Test score: knowledge of pruning, black pod management, farm sanitation, 
shade management, pest management and post harvest operations 


David, 2007 

* Test score: knowledge of tree physiology, disease/ pest management, rational 
pesticide use and post- harvest operations 

* Percentage of farmers that adopt practices (pooled variable of pruned cocoa trees, 
adequate frequency of phyto- sanitary harvesting, managed shade of other trees, spray 
fungicide to moisten pods without run-off) 

De J ager et al., 2009 

Number of known technologies: average number technologies per farm 
mentioned to address soil decline 

* Number of new soil fertility management practices adopted 
Number of new commercial activities 

Dinpanah et al., 20 10 

* Test score: knowledge of biological control (score range 0- 32) 

* Index of adoption of biological control (score range 0-40) 

Endalew, 2009 

* Test score: mean knowledge of practices (score range 0- 16) 

* Index of adoption of coffee management practices (score range 0- 13) 

Erbaugh et al., 2010 

* Test score: pooled variable of mean knowledge on maize, beans, sorghum, 
cowpea, groundnuts (iganga) and groundnuts (kumi) 


Friis- Hansen et al., 2004 


Probability crops sprayed. 


* Probability of adopting IPPM fertiliser techniques (pooled variable of number of 
practices adopted: stopped burning, use of green manure, incorporated other residues, 
used compost, used chicken manure, planted green manure, used chemical fertiliser, 
used cattle manure to improve soil, fallowed to improve soil, mulched to improve soil 
fertility) 

* Probability of adopting IPPM pesticides techniques (pooled variable of number of 

practices adopted: used improved seeds, used the natural enemy to destroy the pest, 
improved soil fertility, monitored pest population, prepared the seed bed early enough, 
monitored weed population, sprayed the crops, did nothing to destroy the pests) 
Probability of adopting other IPPM techniques (pooled variable of number of practices 
adopted: contour ploughing, planted grass strips, planted cover crops, mulched, made 
terraces fanya juu or fnay chini, stopped removing plant residues) 


Gockowski et al., 2005 

* Number of prunings conducted in the 2004 season 

Gockowski et al., 2010 

Fertiliser use (kg applied in the cocoa season 2004- 05) 

Number of positive agricultural practices adopted (pooled variable of number of 
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Study 

Knowledge variable used 

Other adoption variable used 

prunings, number of weedings conducted during the season 2004- 05) 

Godtland et al., 2004 

* Test score: knowledge of I PM practices based on FFS curriculum (score range 
not specified) 


Hiller etal., 2009 

* Test score: mean knowledge of good agricultural practices (score range 0- 10) 

* Percentage of farmers who implemented good agricultural practices (pooled variable) 
Change in plucking frequency 

Huan et al., 1999 

* Test score: composite index of knowledge of pesticide management practices 



(estimated from the following beliefs: applying insecticides will increase yields, 
killing natural enemies can cause more pests, leaf folders in first 40 DAS can 
cause severe damages, leaf folders in first 40 DAS can cause yield loss, spraying 


insecticides for leaf folders has to be done in early season) (score range 15- 5) 
[index rescaled to show a positive effect when SMD>0] 


Islam et al., 2006 

* Probability: percentage of individuals with high level of knowledge on I PM 
including knowledge on pest identification, pest management, insecticides and 
environmental awareness 

* Percentage of individuals with high adoption of technology (as defined when adopts 
67- 100% of the 10 major IPM component technologies) 

Kelemework, 2005 

* Probability: improved knowledge on late blight (pooled variable across all 
education groups) 

* Probability farmers used improved varieties (pooled variable across all education 
groups) 

Khan etal., 2007 

* Test score: growth in mean knowledge (pooled variable of decision-making, 
experimentation, biodiversity and social recognition test scores) (score range 0- 
100) 

Field observations (hours per season) 

Lund etal., 2010 

* Probability: percentage of farmers aware of beneficial insects 


Mandni &Jiggins, 2008 

* Test score: change over time in mean score of knowledge on identification 
score, functional score and ecology score 


Mariyono, 2007 


Percentage of IPM technologies adopted including pest- resistant varieties, technical 
culture, mechanical rat control, chemical insect control, observation 

Mauceri et al., 2007 


* Average number of IPM practices adopted (score range 0-17) 

Mutandwa &Mpangwa, 2004 

* Test score: mean score on technical knowledge (score range 0- 100) 


Norvell &Hammig, 1999 


* Mean score of sustainability index, based on adoption of IPM practices (score range 
0-100) 

Odeyemi et al., 2005 

* Variable definition is not reported 


Orozco-Qrilo et al., 2008 

* Test score: change over time of knowledge of management of water (scoring 
system not reported) 

* Growth in technology adoption index 

Price, 2001 

* Test score: change over time of mean score of entomological knowledge, insect- 
plant interaction and insect knowledge/ practice 


Raoetal., 2012 

* Test score: integrated soil and nutrition management (score range 0- 27) 

* Index of adoption of integrated soil and nutrition management techniques (score range 
0-27) 

Rebaudo & Dangles, 2011 

* Test score: IPM knowledge (score range 0- 5) 
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Knowledge variable used 


Other adoption variable used 


Reddy & Suryamani, 2005 
Rejesusetal., 2010 


Ricker- Gilbert et al., 2008 


Rolaetal., 2002 
Rusike et al., 2004 
Todo &Takahashi, 2011 


Torrez et al., 1999 
Tripp etal., 2005 


Van de Fliert, 2000 


Van Rijn, 2010 


Waartsetal., 2012 
Zuger, 2004 


* Test score: change over time in mean score on cotton pest and other crop 

management practices (score range 0-100) 

* Test score: growth rate of percentage of correctly answered questions 

* Test score: knowledge of simple practices (score range 0- 5), intermediate 

practices (score range 0- 8) and complex practices (score range 0- 3) [FFS 
neighbour group only] 

* Test score: mean scores of pest management tests (pooled variable of nutrient 

management, seed health, pest resistant variety and certified seeds) 

* Test score: mean score of knowledge of soil fertility and water management 

technologies (score range 0- 100) 


* Probability: percentage of people that correctly answer each of 8 I PM questions 

(pooled variable) 

* Probability: percentage of farmers that correctly answer 4 insecticide use 
decision rules (pooled variable of answers to each) 

Number of known technologies: : number of natural enemies named 


* Test score: mean score on knowledge of good agricultural practices (score range 

0 - 10 ) 

Test score: mean score on I PM knowledge (score range not reported) 


* Number of 1) simple, 2) intermediate and 3) complex IPM practices adopted (pooled 
estimate) 


* Percentage of farmers reporting changed practices on any of the following: soil fertility, 

soil and water conservation, new varieties and planting methods 

* Probability farmer adopted positive agricultural practice (pooled variable of probability 

of adopting: farm management, measure/ compost, utilisation of small land, pest 
control, periodic observation/ regular farm visit, proper use of fertiliser, preparation of 
seed bed, proper spacing, soil preparation, weeding, vegetable production, seed 
preparation before sowing, new varieties, appropriate sowing method) 


* Time spent in field inspection (hours per acre per week) 


Percentage of farmers using fertilisers (pooled variable of N- fertiliser, P- fertiliser, K- 
fertiliser and organic fertiliser) 

Percentage of farmers that practice other beneficial agricultural practices (pooled 
variable of seed selection, good water management, field sanitation after experiencing 

weevil attack) 

* Probability farmer adopted positive agricultural practices (pooled variable of 
probability of adopting: shade management, application of organic material, use of 
chemical, organic fungicides, organic fertiliser, organic pesticides, growing seedling in a 
bag, management of seedling, pruning, selective harvest) 


Note: * standardised mean difference (SMD) effect size and standard error calculable. In all other cases, effect size or standard errors were not calculable and therefore these results are excluded from meta- 
analysis. 
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Table E3 Detailed outcomes variables: Pesticide use and agricultural outcome measures 


Study 

Pesticide use measure 

Agricultural outcome measure 

Ali and Sharif, 2012 

* Pesticide use (litres per acre) 

* Yield (kg per ha) 

Amera, 2009 

* Pesticide use: percentage of individuals that used pesticides (pooled 
variable of endosulfan, DDT, malathion, black powder, deltamethrin use) 


Birthal et al., 2000 

Pesticide use (g per ha) 

* Pesticide costs (Rs, unclear if per ha) 

Yield (quintal per ha) 

* Value of output (Rs per household) (unclear if used current or constant Rs) 

Carlberg et al., 20 12 


* Yield ( 50kg bags per acre 20 10) 

Cavatassi et al., 2011 

* Pesticide use (kg per ha insecticide applied) 

* Yield (kg per ha) 

* Net revenue: gross margin (USD per ha) (unclear if used current or constant USD) 

Chi et al., 1999 

Pesticide use: Mean number of insecticide sprays during 0-40 days after 
sowing 


Coleetal., 2007 

* Pesticide use (kg per ha) 


DANIDA, 2011 


Annual income (BD taka per household) 

Davis etal., 2012 


* Growth rate in value of yield (growth rate in value in local currency per acre) 

Dinpanah et al., 20 10 


* Yield (ton per ha) 

* Income (USD per year) (unclear if used current or constant USD and if income is per 
unit of land) 

Feder et al., 2004 

* Growth rate in pesticide costs (Rp per ha, 1998 prices) in the main rice 
plot 

* Growth rate in yield (growth rate, kg per ha) 

Gockowski et al., 2005 

Pesticide use (kg per ha copper sulfate applied in 2004) 

Yield (kg per ha 2004) 

Gockowski et al., 2010 


* Quantity of produce sold in 2004- 05 season 

Haiyang, 2002 

Pesticide use (gper ha of chemical pesticides, fungicides, herbicides and 
rodentiddes) 

Pestidde costs (USD per ha) (total expenditure on chemical pestiddes, 
fimgiddes, herbiddes, rodentidde) 

Yield (kg per ha) 

Net income (USD per ha) (unclear if this is only for rice production; unclear if used 
current or constant USD) 

Hiller etal., 2009 


* Growth in yield (kg per acre) 

Huan et al., 1999 

* Pestidde use (number of insectidde sprays per season) 

* Yield (ton per ha) (unclear if this is only for rice production) 

Islam et al., 2006 

Pestidde costs (taka per ha) 

Yield (kg per ha) 

J ones, 2002 

Pestidde costs (Rs per ha) 

Yield (kg per ha) (pooled variable of aubergine, chilli and potato yields) 

Net revenue (value per ha) (pooled variable of aubergine, chilli and potato) 

Kabir & Uphoff, 2007 


Growth of yield (ton per ha) 

Khalid, n.d. 

* Pestidde use (mL per feddan) 


Khan etal., 2007 

* Growth in pestidde use (kg per ha insectidde) 

* Growth in pestidde costs (USD per ha) 

* Growth of yield (kg per ha) 

Growth in net revenue: gross margin (USD per ha) 

Labarta, 2005 

* Pestidde use 

* Yield 

* Net revenue 

Lama etal., 2003. 


Net revenue (N Rp per ha) 
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Study 

Pesticide use measure 

Agricultural outcome measure 

Mandni &Jiggins, 2008 

* Growth rate of pesticide use (ml a.i. per ha) (pooled variable of highly 
toxic and less harmful pesticide) 

Growth rate of yield (kg per ha) (unclear if this is only for cotton production) 

Murphy et al., 2002 

* Pesticide use (number of times sprayed per farmer per year) 


Mutandwa &Mpangwa, 2004 

* Pesticide costs (currency amount per acre) 

* Yield (number of bales) (unclear if bales is per any unit of land) 

Naiketal., 2010 


* Yield (quintals of produce) 

* Net income (Rs.) 

Ooi &Kenmore, 2005 

Growth in pesticide cost (USD) (unclear if relative to any unit of land) 

Growth in revenue: gross margin (USD per hectare) 

Orozcx) Qrilo et al., 2008 


* Growth in yield (ton per ha) 

Palis, 1998 

Growth in insecticide use and herbicide use (kg per ha) (pooled variable 
across diy and wet season) 

* Growth in yield (ton per ha) (pooled variable of diy season and wet season) 

* Growth in net revenues (Ps. per ha) (pooled variable of diy season and wet season) 

Pananurak, 2010 

* Growth in pesticide costs (USD per ha) 

* Growth in yield (kg per ha) 

* Growth in net revenue: gross margin (USD per ha) 

Pande et al., 2009 


* Yields (ton per ha) 

Net income (Rs.) over 5 years 

Praneetvatakul &Waibel, 2006 

* Growth in pesticide costs (total expenditure over reference period) 


Price, 2001 

Pesticide use (number of insecticide sprays in 1997) 
Pesticide cost (Ps. per ha insecticide) 


Rejesusetal., 2010 

* Growth rate in pesticide use (total number of insecticide sprays per ha) 

* Growth rate in yields (tonnes per ha) 

Todo &Takahashi, 2011 


* Growth rate in value of production (Eth birr) 

Torrez et al., 1999 


Yield (no description of the outcome provided) 

Tripp etal., 2005 

Pesticide use: number of insecticide applications (2002-03) 


Van de Fliert, 2000 

Pesticide use: number of pesticide applications per season 

Yield (kg per ha) (unclear if this only potato yields) 

Van den Berg and Amarasinghe, 
2002 

* Pesticide cost: total expenditure on fertilisers, insecticides, hmgiddes and 
herbicides (USD per ha) 

* Yield (kg per ha) 

* Profit (USD per ha) 

Van Rijn, 2010 


* Yield (kg per ha, 2007) 

* Net revenue ( Soles per kg per ha) 

Waartsetal., 2012 

Growth in pesticide cost (K Sh per bush) 

Growth in yield (kg per acre) 
Net income (KSh.) 

Walter- Echols &Soomro, 2005 

Growth in pesticide use (kg per ha) 


Wandji etal., 2007 


* Sales: Kg of cocoa sold in the 2004- 05 season 

Williamson et al., 2003 

Pesticide cost: cash outlay on insecticide sprays, pheromone sprays, 
pheromone traps, preventative fungicide sprays, chilli extract sprays, 
curative fungicide sprays (USD per ha) 

Net revenue: gross margin (USD per ha) 
Gross income (USD per ha) 

Wu, 2010 

* Growth in insecticide cost (USD per ha) 

* Growth rate in yield (kg per ha) 

Yamazaki and Resosudarmo, 2008 

* Growth in pesticide cost (Rp per ha, 1998 prices) in the main rice plot 


Yang etal., 2005 

* Pesticide use (litres per ha) 

* Yield (kg per ha) 
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Study 

Pesticide use measure 

Agricultural outcome measure 


* Pesticide costs (USD per household) 

* Net revenue: gross margin (USD per ha) 


Yorobeetal., 2011 

* Pesticide costs (pesos per ha) 


Zuger, 2004 

Pesticide use (number of applications in season 2000- 01 

Yield (ton per ha) 


Note: * response ratio effect size and standard error calculable. In all other cases, effect size or standard errors were not estimatable and therefore these results are excluded from meta- analysis. 
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Table E4 Detailed outcomes variables: environment health and empowerment outcomes 


Study 

Environment outcomes 

Health outcomes 

Empowerment outcomes 

Amera, 2009 


* Probability of pesticide poisoning 


Cavatassi et al., 2011 

* Change in environmental impact quotient 
(EIQ**) score 



DANIDA, 2011 


Annual expenditure on health (Taka per 
household) 


Friis-Hansen et al., 2004 

* Change in soil fertility 



Friis- Hansen &Duveskog, 2012 



Pooled variable of self-reported empowerment indices: 
innovation uptake, access to services, engaging with markets 
and collective action/ social relations 

Hiller et al., 2009 



Change over time in perception of empowerment (further 
detail in the definition of empowerment is not reported) 

Labarta, 2005 


* Incidence of respiratory difficulties, eye 
irritation, stomach ache, blurred vision 


Pananurak, 2010 

* Growth rate of environmental impact quotient 
index (EIQ**) score 



Praneetvatakul &Waibel, 2006 

* Growth rate of environmental impact quotient 
index (EIQ**) score 



Rusike et al., 2004 



Pooled variable of share of famers reporting better position 
to solve local agricultural problems on their own and share 
reporting making new demands on extension staff 

Van Rijn, 2010 



* Pooled variable of probability of the farmers positively 
answering self-esteem questions: feeling capable of solving 
problems in the field, feeling comfortable in giving an 
opinion, participating in the community 

Walter- Echols &Soomro, 2005 

Change over time in environmental impact 
quotient (EIQ**) score 




Notes: * response ratio effect size and standard error calculable. In all other cases, effect size or standard errors were not estimatable and therefore these results are excluded from 
meta- analysis. ** EIQ calculation uses active ingredients of pesticides and applies a rating system in ten categories to identify a single value of the environmental impact rating. The ten 
categories include: (i) action mode of pesticides, (ii) acute toxicity to birds, (iii) fish, (iv) bees, (v) acute dermal toxicity, (vi) long-term health effects, (vii) residue half-life in soil and 
(viii) plant surface, (ix) toxicity to beneficial organisms, and (x) groundwater and runoff potential (Kovach et al., 1992). 
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Appendix F 


RESULTS OF CRITICAL APPRAISAL: IMPACT EVALUATIONS 


Study 


Achonga et al., 2011 


Ali & Sharif, 2012 


Amera, 2009 


Bentley et al., 2007 


Birthal et al., 2000 


Bunyatta et al., 
2006 


Study design Selection bias and Spillovers and Outcome reporting Analysis reporting Other sources of bias Overall risk of 

(analysis method) confounding contamination bias addressed? bias addressed? addressed? bias assessment 

addressed? addressed? 


Cross-section 

(univariate 

regression) 


Cross- section (PSM) 



1 =Yes, 2 =No, 8 = Undear 


High risk 


Medium risk 


Cross- section 


Cross- section 


Cross-section 


Cross-section 



High risk 


High risk 


High risk 


High risk 


Unit of analysis 
assessment 


N/A 


Unclear 


Unclear 


Unclear 


Unclear 


High probability of 
error (UoA error 
correction applied) 
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Study 


Carlberg et al., 20 12 


Cavatassi et al., 
2011 


Chi et al., 1999 


Coleetal., 2007 


DAMDA. 2011 


Dankyi etal., 2005 


David & Asamoah, 
2011 


David, 2007 


Davis etal., 2012 
(Kenya) 


Davis etal., 2012 
(Tanzania) 


Davis etal., 2012 
(Uganda) 


De J ager et al., 
2009 


Study design Selection bias and Spillovers and Outcome reporting Analysis reporting Other sources of bias Overall risk of 

(analysis method) confounding contamination bias addressed? bias addressed? addressed? bias assessment 

addressed? addressed? 


1 =Yes, 2 =No, 8 = Unclear 


Cross-section (IV 
regression) 


Cross-section (PSM- 
WLS regression) 


8 


8 



1 


1 



8 


8 


High risk 


Medium risk 


Cross-section 


Cross-section 


Cross-section (PSM) 


Cross-section 


Cross-section 

(regression) 


Cross-section 


Longitudinal 
(covariate matching) 



1 


8 


1 


8 


1 


1 


8 


Longitudinal 8 8 8 

(covariate matching) 



8 


1 



High risk 


High risk 


High risk 


High risk 


High risk 


High risk 


Medium risk 


Medium risk 


High risk 


High risk 


Unit of analysis 
assessment 


Low probability of 
error 


Low probability of 
error 


Unclear 


Unclear 


N/A 


Unclear 


Unclear 


N/A 


Low probability of 
error 


Low probability of 
error 


Low probability of 
error 


Unclear 
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Study 


Dinpanah et al., 
2010 


Endalew, 2009 


Erbaugh et al., 2010 


Feder et al., 2004 


Friis- Hansen & 
Duveskog, 2012 


Friis- Hansen & 
Duveskog, 2012 


Friis- Hansen & 
Duveskog, 2012 


Friis-Hansen et al., 
2004 


Gockowski et al., 
2005 


Gockowski et al., 
2010 


Godtland et al., 
2004 


Haiyang, 2002 


Study design Selection bias and Spillovers and Outcome reporting Analysis reporting Other sources of bias Overall risk of 

(analysis method) confounding contamination bias addressed? bias addressed? addressed? bias assessment 

addressed? addressed? 


Cross-section 


Cross- section 


Cross-section 


Longitudinal (DID 
regression) 


Cross-section 


Cross-section 


Cross-section 


Cross-section 


Cross-section 


Cross-section 

(regression) 


Cross- section ( PSM) 


Cross-section 



8 


1 


8 


8 


8 


8 


8 


8 


8 


8 


1 


8 


1 =Yes, 2 =No, 8 = Unclear 



8 


8 


8 


8 


8 


8 


8 


8 


8 


8 


8 


8 


8 


8 


High risk 


High risk 


High risk 


Medium risk 


High risk 


High risk 


High risk 


High risk 


High risk 


High risk 


Medium risk 


High risk 


Unit of analysis 
assessment 


Unclear 


Unclear 


Unclear 


Low probability of 
error 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


Unclear 


289 


Study 


Hiller et al., 2009 


Huan et al., 1999 


Islam et al., 2006 


J ones, 2002 


Kabir &Uphoff, 
2007 


Kelemework, 2005 


Khalid, n.d. 


Khanetal., 2007 


Labarta, 2005 


Lamaetal., 2003 


Lundetal., 2010 


Mandni & J iggins 
2008 


Study design Selection bias and Spillovers and Outcome reporting Analysis reporting Other sources of bias Overall risk of 

(analysis method) confounding contamination bias addressed? bias addressed? addressed? bias assessment 

addressed? addressed? 


1 =Yes, 2 =No, 8 = Unclear 


Cross-section 
(knowledge) / 
longitudinal 
(adoption, yields) 
Cross-section 


Cross-section 


Cross-section 


Cross-section 


Longitudinal 


Cross-section 


Longitudinal 


Cross-section (IV 
regression) 


Cross-section 


Cross-section 


Longitudinal 



8 


8 


8 


8 


8 


8 


8 


8 


8 


8 


8 


8 


High risk 


High risk 


High risk 


High risk 


High risk 


High risk 


High risk 


High risk 


High risk 


High risk 


High risk 


High risk 


Unit of analysis 
assessment 


Unclear 


Unclear 


N/A 


Unclear 


N/A 


Unclear 


Unclear 


Unclear 


Low probability of 
error 


Unclear 


N/A 


Unclear 
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Study 


Study design Selection bias and Spillovers and Outcome reporting Analysis reporting Other sources of bias Overall risk of Unit of analysis 

(analysis method) confounding contamination bias addressed? bias addressed? addressed? bias assessment assessment 

addressed? addressed? 


1 =Yes, 2 =No, 8 = Unclear 


Mangan &Mangan, Longitudinal 
1998 


Mariyono, 2007 Longitudinal 


Mauceri et al., 2007 Cross-section (IV 
regression) 


Maumbe& 
Swinton, 2003 


Cross-section 

(regression) 


Murphy et al., 2002 Cross- section 


Mutandwa & 
Mpangwa, 2004 


Cross-section 


Naik et al. , 20 10 Cross- section 


Norvell & Harnmig, Cross- section 

1999 (regression) 


Odeyemi et al., 
2005 


Cross-section 


Olanya, 2010 


Cross-section 


Ooi and Kenmore, Longitudinal 
n.d. 
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Study 


Orozco Cirilo et al., 
2008 (yields) 


Palis, 1998 


Pananurak, 2010 
(China) 


Pananurak, 2010 
(India) 


Pananurak, 2010 
(Pakistan) 


Pandeetal., 2009 


Pouchepparadjou et 
al., 2005 


Praneetvatakul & 
Waibel, 2006 


Price, 2001 


Raoetal., 2012 


Rebaudo & Dangles, 
2011 


Reddy & 
Suryamani, 2005 


Study design Selection bias and Spillovers and Outcome reporting Analysis reporting Other sources of bias Overall risk of 

(analysis method) confounding contamination bias addressed? bias addressed? addressed? bias assessment 

addressed? addressed? 


1 =Yes, 2 =No, 8 = Unclear 


Longitudinal 
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Longitudinal (DID 
regression) 


Longitudinal (DID 
regression) 
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8 Medium risk 
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Medium risk 
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Unit of analysis 
assessment 
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N/A 


Unclear 
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Study 


Rejesus et al., 2010 


Ricker- Gilbert et al., 
2008 


Rolaetal., 2002 


Rusike et al., 2004 


Todo &Takahashi, 
2011 


Torrez et al., 1999 


Tripp etal., 2005 


Van de Fliert, 2000 


Van den Berg and 
Amarasinghe, 2002 


Van Rijn, 2010 


Waartsetal., 2012 


Walter- Echols & 
Soomro, 2005 


Study design Selection bias and Spillovers and Outcome reporting Analysis reporting Other sources of bias Overall risk of 

(analysis method) confounding contamination bias addressed? bias addressed? addressed? bias assessment 

addressed? addressed? 


1 =Yes, 2 =No, 8 = Unclear 


Longitudinal (DID 
regression) 
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regression) 
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N/A 


Unclear 


Unclear 
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Study 

Study design 
(analysis method) 

Selection bias and 
confounding 
addressed? 

Spillovers and 
contamination 
addressed? 

Outcome reporting 
bias addressed? 

Analysis reporting 
bias addressed? 

Other sources of bias 
addressed? 

Overall risk of 
bias assessment 

Unit of analysis 
assessment 




i= 

=Yes, 2 =No, 8 = Unclear 




Wandji etal., 2007 
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(regression) 
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8 
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RESULTS OF CRITICAL APPRAISAL: QUALITATIVE EVALUATIONS 
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Appendix G 


META-ANALYSES BY PROGRAMME NAME 


Figure A18 Knowledge outcomes by programme name 


Study 

ID 

FFS participants 

Kenya: Kenya Tea Development Agency/Lipton Sustainabile Agriculture Project 

Peru: CIP CARE Andean FFS 

Vietnam: FAO Programme for Community IPM in Asia 

Ghana: National Cocoa Disease and Pest Control Program (CODAPEC) 

Cameroon: Sustainable Tree Crops Program (STCP) Phase II 

Ethiopia: Jimma and Sidama FFS 

Mexico: El Proyecto Manejo Sustentable de Laderas (PMSL) 

Philippines: Central Luzon (IRRI, Philrice) 

India: Sree Ram Sagar Project (SRSP), Andhra Pradesh 
India: FAO/EU IPM Programme for Cotton in Asia 
Pakistan: Cotton IPM programme (FAO, EU, GoP) 

Ethiopia: Integrated Management of Potato Late Blight (CIP, IFAD) 

Zimbabwe: Cotton IPM-FFS, Sanyati district 

Iran: FFS programmes 

Pakistan: National IPM Programme, Khairpur 

Kenya: KARI Farmer Field School Project 

Uganda: IPM Collaborative Research Support Program (CRSP) 

Ecuador: unclear potato IPM (CIP) 

Subtotal (l-squared = 93.2%, p = 0.000) 

FFS neighbours 

Pakistan: National IPM Programme, Khairpur 

Mexico: El Proyecto Manejo Sustentable de Laderas (PMSL) 

India: FAO/EU IPM Programme for Cotton in Asia 
Bangladesh: National IPM Programme (FAO) 

Ecuador: unclear potato IPM (CIP) 

Subtotal (l-squared = 5.2%, p = 0.377) 


Standardised mean difference 



ES (95% Cl) 

0.03 

(-0.03, 0.08) 

0.14 

(0.03, 0.25) 

0.17 

(0.03,0.31) 

0.17 

(0.07, 0.27) 

0.20 

(0.17,0.24) 

0.27 

(-0.06, 0.60) 

0.42 

(0.37, 0.47) 

0.42 

(-0.17, 1.01) 

0.43 

(-0.02, 0.87) 

0.45 

(-0.04, 0.94) 

0.46 

(0.18, 0.73) 

0.54 

(-0.22, 1.29) 

0.59 

(0.25, 0.92) 

0.67 

(0.41,0.92) 

0.79 

(0.29, 1 .29) 

1.03 

(0.65, 1.41) 

1.14 

(0.93, 1.34) 

1.79 

(1.17,2.41) 

0.46 

(0.33, 0.58) 

-0.13 (-0.68, 0.42) 

-0.07 (-0.07, -0.06) 

0.05 

(-0.45, 0.56) 

0.17 

(-0.25, 0.59) 

0.38 

(-0.15, 0.91) 

-0.05 (-0.13, 0.03) 


T 


r 

.5 1 2 

Favours FFS participants 


Favours non-FFS 
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Figure A19 Pesticide use by programme name 


Study 

ID 


FFS participants 

Indonesia: National IPM Training Project Phase II (FAO, World Be flak, G el ) 

India: Maharashtra project (National Centre for Integrated Pest Management) 

Philippines: IPM Collaborative Research Support Program (CRSP), Nueva Ecija (USAID) - 
China: FAO/EU IPM Programme for Cotton in Asia, Hubei Province 
Sudan: Gezira Scheme, FAO IPM in Vegetables 
Vietnam: FAO Programme for Community IPM in Asia 
India: FAO/EU IPM Programme for Cotton in Asia 
Zimbabwe: Cotton IPM-FFS, Sanyati district 
Pakistan: FAO/EU IPM Programme for Cotton in Asia 
Kenya: Cotton IPM project (EU & PAN-UK) 

India: FAO/EU IPM Programme for Cotton in Asia 
China: FAO/EU IPM Programme for Cotton in Asia 
Sri Lanka: IPVM Project (FAO, UNEP) 

Thailand: FAO/EU IPM Programme for Cotton in Asia 
Vietnam: FAO National IPM Program, Nam Dinh 
Ecuador: EcoSalud 

Pakistan: Cotton IPM programme (FAO, EU, GoP) 

Pakistan: National IPM Programme, Khairpur 
Nicaragua: Project for IPM in Central America (PROMIPAC) 

Indonesia: National IPM Training Project Phase II (FAO, World Bank, Gol) 

Ecuador: Plataformas Program (FAO) 

Uganda: East African Sub-regional Pilot Project Phase II (FAO) 

Subtotal (l-squared = 93.7%, p = 0.000) 


FFS neighbours 

India: FAO/EU IPM Programme for Cotton in Asia 

Indonesia: National IPM Training Project Phase II (FAO, World Bank, Gol) 

China: FAO/EU IPM Programme for Cotton in Asia 

Pakistan: FAO/EU IPM Programme for Cotton in Asia 

Nicaragua: Project for IPM in Central America (PROMIPAC) 

Thailand: FAO/EU IPM Programme for Cotton in Asia 
Pakistan: National IPM Programme, Khairpur 

Indonesia: National IPM Training Project Phase II (FAO, World Bank, Gol) 
Subtotal (l-squared = 85.2%, p = 0.000) 



Favours FFS participants 


Favours non-FFS 


Response ratio 

ES (95% Cl) 


0.20 

(0.01, 

3.23) 

0.21 

(0.17, 

0.26) 

0.37 

(0.18, 

0.78) 

0.41 

(0.36, 

0.46) 

0.48 

(0.31 , 

0.75) 

0.52 

(0.24, 

1.12) 

0.52 

(0.30, 

0.92) 

0.57 

(0.36, 

0.89) 

0.59 

(0.41 , 

0.87) 

0.61 

(0.52, 

0.71) 

0.67 

(0.46, 

0.97) 

0.71 

(0.64, 

0.80) 

0.82 

(0.74, 

0.90) 

0.82 

(0.68, 

0.98) 

0.83 

(0.75, 

0.93) 

0.88 

(0.68, 

1.13) 

0.90 

(0.75, 

1.09) 

0.91 

(0.28, 

2.94) 

0.95 

(0.39, 

2.34) 

1.30 

(1.08, 

1.57) 

1.34 

(0.99, 

1.80) 

1.42 

(1.09, 

1.86) 

0.69 

(0.57, 

0.84) 


0.54 

(0.25, 

1.15) 

0.67 

(0.12, 

3.88) 

0.68 

(0.62, 

0.76) 

0.78 

(0.40, 

1.49) 

0.99 

(0.42, 

2.33) 

1.15 

(0.92, 

1.43) 

1.20 

(0.40, 

3.53) 

1.30 

(1.09, 

1.55) 

0.91 

(0.66, 

1.26) 


Figure A20 Other adoption measures by programme name 


study Standardised mean difference 

ID ES (95% Cl) 


Peru: Cajamarca FFS (CARE, CIP) 

Ethiopia: Participatory Forest Management Project in the Belete-Gera Regional Forest Priority Area 
Indonesia: National IPM Training Project Phase II (FAO, World Bank, Gol) 

Uganda: East African Sub-regional Pilot Project Phase II (FAO) 

India: Sree Ram Sagar Project (SRSP), Andhra Pradesh 
Pakistan: Cotton IPM programme (FAO, EU, GoP) 

Ethiopia: Jimma and Sidama FFS 

Peru: DE Foundation Coffee Project 

Mexico: El Proyecto Manejo Sustentable de Laderas (PMSL) 

Ethiopia: Integrated Management of Potato Late Blight (CIP, IFAD) 

Bangladesh: National IPM Programme (FAO) 

Kenya: Cotton IPM project (EU & PAN-UK) 

Ecuador: IPM Collaborative Research Support project (CRSP), Carchi 

Kenya: KARI Farmer Field School Project 

Iran: FFS programmes 

Overall (l-squared = 99.6%, p = 0.000) 



-0.41 (-0.71,-0.11) 
0.06 (-0.06,0.19) 
0.08 (0.07, 0.09) 
0.10 (-0.14, 0.33) 
0.11 (-0.33,0.55) 
0.22 (0.17,0.27) 
0.24 (-0.09, 0.58) 
0.46 (0.23, 0.69) 
0.64 (0.58, 0.70) 
0.70 (-0.28, 1.67) 
0.92 (0.29, 1.55) 
1.19(1.15, 1.23) 
1.41 (0.96, 1.86) 
1.45(1.04, 1.85) 
1.46(1.18, 1.73) 
0.56 (0.26, 0.86) 


Favours non-FFS 


0 .5 1 2 

Favours FFS participants 
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Figure A2 1 Yields by programme name 


Study 

ID 


Response ratio 

ES (95% Cl) 


FFS participants 

India: FAO/EU IPM Programme for Cotton in Asia 

Peru: DE Foundation Coffee Project — 

India: Karnataka Community Based Tank Management Project (KCBTMP) 

Nicaragua: Project for IPM in Central America (PROMIPAC) 

Vietnam: FAO Programme for Community IPM in Asia 

Indonesia: National IPM Training Project Phase II (FAO, World Bank, Gol) 

China: FAO/EU IPM Programme for Cotton in Asia 
Pakistan: Cotton IPM programme (FAO, EU, GoP) 

Ghana: National Cocoa Disease and Pest Control Program (CODAPEC) 

China: FAO/EU IPM Programme for Cotton in Asia, Hubei Province 

Kenya: Kenya Tea Development Agency/Lipton Sustainabile Agriculture Project 

Pakistan: National IPM Programme, Khairpur 
Ecuador: Plataformas Program (FAO) 

Tanzania: East African Sub-regional Pilot Project Phase II (FAO) 

India: Maharashtra project (National Centre for Integrated Pest Management) 

Pakistan: FAO/EU IPM Programme for Cotton in Asia 
Iran: FFS programmes 

Cameroon: Sustainable Tree Crops Program (STCP) Phase II 

Zimbabwe: Cotton IPM-FFS, Sanyati district 

Philippines: BIPM project Central Luzon (IRRI, FAO, Philrice) 

Peru: Cajamarca FFS (CARE, CIP) 

Ghana: Peanut Collaboration Research Support Program (CRSP) 

Indonesia: National IPM Training Project Phase II (FAO, World Bank, Gol) 

Sri Lanka: IPVM Project (FAO, UNEP) 

Kenya: East African Sub-regional Pilot Project Phase II (FAO) 

Nepal: ICM FFS 

Mexico: El Proyecto Manejo Sustentable de Laderas (PMSL) 

Ethiopia: Participatory Forest Management Project in the Belete-Gera Regional Forest Priority Area 
Subtotal (l-squared = 91 .4%, p = 0.000) 


FFS neighbours 

India: FAO/EU IPM Programme for Cotton in Asia 
Pakistan: National IPM Programme, Khairpur 

Indonesia: National IPM Training Project Phase II (FAO, World Bank, Gol) 
Nicaragua: Project for IPM in Central America (PROMIPAC) 

China: FAO/EU IPM Programme for Cotton in Asia 
Pakistan: FAO/EU IPM Programme for Cotton in Asia 
Indonesia: National IPM Training Project Phase II (FAO, World Bank, Gol) 
Subtotal (l-squared = 53.3%, p = 0.045) 



0.80 

(0.61, 

1.05) 

0.86 

(0.63, 

1.18) 

0.89 

(0.83, 

0.96) 

0.97 

(0.92, 

1.02) 

0.97 

(0.72, 

1.31) 

0.98 

(0.96, 

1.01) 

1.08 

(1.03, 

1.14) 

1.09 

(1.03, 

1.15) 

1.14 

(1.03, 

1.25) 

1.15 

(0.94, 

1.41) 

1.17 

(0.53, 

2.56) 

1.17 

(0.97, 

1.42) 

1.22 

(0.97, 

1.53) 

1.23 

(1.00, 

1.51) 

1.24 

(1.13, 

1.36) 

1.24 

(1.01, 

1.54) 

1.32 

(1.22, 

1.42) 

1.32 

(1.07, 

1.63) 

1.36 

(1.06, 

1.73) 

1.36 

(0.97, 

1.92) 

1.44 

(1.09, 

1.92) 

1.58 

(1.19, 

2.10) 

1.67 

(1.23, 

2.26) 

1.68 

(1.30, 

2.18) 

1.81 

(1.15, 

2.84) 

2.11 

(1.25, 

3.56) 

2.62 

(2.23, 

3.08) 

*■ 2.71 

(1.11, 

6.60) 

1.23 

(1.15, 

1.33) 

0.79 

(0.63, 

1.00) 

0.97 

(0.74, 

1.26) 

0.99 

(0.97, 

1.01) 

1.00 

(0.99, 

1.01) 

1.03 

(0.99, 

1.08) 

1.03 

(0.86, 

1.25) 

1.43 

(1.05, 

1.96) 

1.00 

(0.98, 

1.03) 


T 


T 


.5 1 2 3 6 


Favours non-FFS 


Favours FFS participants 


Figure A22 Net revenues by programme name 


study Response ratio 

ID ES (95% Cl) 


FFS participants 

Nicaragua: Project for IPM in Central America (PROMIPAC) ^ 

India: FAO/EU IPM Programme for Cotton in Asia 

Kenya: Kenya Tea Development Agency/Lipton Sustainabile Agriculture Project 

China: FAO/EU IPM Programme for Cotton in Asia 

Pakistan: FAO/EU IPM Programme for Cotton in Asia 

India: Karnataka Community Based Tank Management Project (KCBTMP) 

Indonesia: National IPM Training Project Phase II (FAO, World Bank, Gol) 

Sri Lanka: IPVM Project (FAO, UNEP) 

China: FAO/EU IPM Programme for Cotton in Asia, Hubei Province 
Pakistan: National IPM Programme, Khairpur 
Subtotal (l-squared = 57.1%, p = 0.013) 

FFS+input/marketing 

India: Maharashtra project (National Centre for Integrated Pest Management) 
Peru: DE Foundation Coffee Project 
Ecuador: Plataformas Program (FAO) 

Philippines: BIPM project Central Luzon (IRRI, FAO, Philrice) 

Subtotal (l-squared = 96.2%, p = 0.000) 

FFS neighbours 

India: FAO/EU IPM Programme for Cotton in Asia 
China: FAO/EU IPM Programme for Cotton in Asia 
Pakistan: FAO/EU IPM Programme for Cotton in Asia 
Nicaragua: Project for IPM in Central America (PROMIPAC) 

Pakistan: National IPM Programme, Khairpur 
Subtotal (l-squared = 0.0%, p = 0.706) 





0.28 

(0.02, 

3.48) 

1.06 

(0.68, 

1.66) 

1.14 

(0.92, 

1.41) 

1.17 

(1-08, 

1.27) 

1.23 

(1-09, 

1.40) 

1.25 

(1.09, 

1.42) 

1.31 

(1.11, 

1.55) 

1.41 

(1-19, 

1.67) 

1.53 

(1.10, 

2.15) 

3.40 

(1-94, 

5.97) 

1.28 

(1.17, 

1.41) 


1.43 

(1.19, 

1.72) 

2.00 

(1-02, 

3.94) 

3.34 

(1.56, 

7.15) 

4.61 

(3.83, 

5.56) 

2.57 

(1-18, 

5.58) 


0.93 

(0.66, 

1.32) 

1.07 

(1.00, 

1.14) 

1.13 

(1.01, 

1.26) 

1.39 

(0.66, 

2.92) 

1.51 

(0.51, 

4.45) 

1.08 

(1.03, 

1.15) 


.2 .5123 


Favours non-FFS 
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META-ANALYSES INCLUDING ALL STANDARD ERRORS 
CORRECTED FOR POSSIBLE UNIT OF ANALYSIS ERRORS 


Figure A23 Knowledge outcomes corrected for possible unit of analysis error: all studies 


Study 

ID 


Standardised mean difference 

ES (95% Cl) 


FFS participants 
Waarts et al., 2012 (Kenya) 

Godtland et al., 2004 (Peru) 

Rejesus et al., 2010 (Vietnam) 

David & Asamoah, 201 1 (Ghana) 

David, 2007 (Cameroon) 

Endalew, 2009 (Ethiopia) 

Orozco Cirilo et al., 2008 (Mexico) 

Price et al., 2001 (Philippines) 

Rao et al., 2012 (India) 

Reddy & Suryamani, 2005 (India) 

Ali & Sharif, 201 1 (Pakistan) 

Kelemework, 2005 (Ethiopia) 

Mutandwa & Mpangwa, 2004 (Zimbabwe) 
Dinpanah et al., 2010 (Iran) 

Khan et al., 2007 (Pakistan) 

Bunyatta et al., 2006 (Kenya) 

Erbaugh, 2010 (Uganda) 

Rebaudo & Dangles, 2011 (Ecuador) — 
Subtotal (l-squared = 70.8%, p = 0.000) 



0.03 

(-0.03, 0.08) 

0.14 

(-0.06, 0.34) 

0.17 

(-0.16, 0.51) 

0.17 

(-0.03, 0.37) 

0.20 

(0.16, 0.24) 

0.27 

(0.11, 0.43) 

0.42 

(-0.40, 1.24) 

0.42 

(-0.40, 1.24) 

0.43 

(-0.41, 1.26) 

0.45 

(-0.43, 1.34) 

0.46 

(-0.20, 1.11) 

0.54 

(-0.51, 1.59) 

0.59 

(-0.56, 1.73) 

0.67 

(-0.64, 1.97) 

0.79 

(-2.63, 4.22) 

1.03 

(-0.24, 2.30) 

1.14 

(0.73, 1.55) 

— > 1.79 

(-1.71,5.29) 

0.26 

(0.14, 0.37) 


FFS neighbours 

Khan et al., 2007 (Pakistan) < ♦ 

Orozco Cirilo et al., 2008 (Mexico) ■*- 

Reddy & Suryamani, 2005 (India) ♦ 

Ricker-Gilbert et al., 2008 (Bangladesh) — « 

Rebaudo & Dangles, 201 1 (Ecuador) 

Subtotal (l-squared = 0.0%, p = 0.517) <) 


■> -0.13 (-21 .45, 21.19) 
-0.07 (-0.20, 0.06) 
0.05 (-0.05, 0.15) 
0.17 (-0.37, 0.71) 
0.38 (-0.36, 1.11) 
0.02 (-0.06, 0.09) 


-3 -1 0 .5 1 2 5 

Favours non-FFS Favours FFS participants 


Figure A24 Knowledge outcomes corrected for possible unit of analysis error: excluding 
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Study 


Standardised mean difference 


ID 

ES (95% Cl) 

FFS participants 


Godtland et al., 2004 (Peru) 

0.14 (-0.06, 0.34) 

Rejesus et al., 201 0 (Vietnam) — 

0.17 (-0.16, 0.51) 

Ali & Sharif, 201 1 (Pakistan) — 

• 0.46 (-0.20, 1.11) 

Subtotal (l-squared = 0.0%, p = 0.659) 
1 

<(^> 0.17(0.00,0.33) 

1 1 1 


-.5 0 .5 1 2 

Favours non-FFS Favours FFS participants 


Figure A25 Pesticide use corrected for possible unit of analysis error: all studies 


Study 

ID 


FFS participants 

Yamazaki & Resosudarmo, 2007 ( Cnd o nesia ) 
Birthal et al., 2000 (India) < ♦ 

Yorobe et al., 201 1 (Philippines) 

Yang et al., 2005 (China) 

Khalid, n.d. (Sudan) 

Rejesus et al., 2010 (Vietnam) 

Pananurak, 2010 (India) 

Mutandwa & Mpangwa, 2004 (Zimbabwe) 
Pananurak, 2010 (Pakistan) 

Amera, 2008 (Ethiopia) 

Mancini et al., 2008 (India) 

Wu, 2010 (China) 

Van den Berg et al., 2002 (Sri Lanka) 
Praneetvatakul & Waibel, 2006 (Thailand) 
Murphy et al., 2002 Vietnam) 

Cole et al., 2007 (Ecuador) 

Ali & Sharif, 201 1 (Pakistan) 

Khan et al., 2007 (Pakistan) < 

Labarta, 2005 (Nicaragua) 

Feder et al., 2004 (Indonesia) 

Cavatassi et al., 201 1 (Ecuador) 

Friis-Hansen et al., 2004 (Uganda) 

Subtotal (l-squared = 90.3%, p = 0.000) 


FFS neighbours 
Pananurak, 2010 (India) 

Yamazaki & Resosudarmo, 2007 (In don e sia) 
Wu, 2010 (China) 

Pananurak, 2010 (Pakistan) 

Labarta, 2005 (Nicaragua) 

Praneetvatakul & Waibel, 2006 (Thailand) 

Khan et al., 2007 (Pakistan) < 

Feder et al., 2004 (Indonesia) 

Subtotal (l-squared = 85.0%, p = 0.000) 



■> 



Response ratio 
ES (95% Cl) 

0.20 

(0.01, 

3.23) 

0.21 

(0.10, 

0.46) 

0.37 

(0.12, 

1.15) 

0.41 

(0.36, 

0.46) 

0.48 

(0.23, 

1.00) 

0.52 

(0.24, 

1.12) 

0.52 

(0.26, 

1.05) 

0.57 

(0.36, 

0.89) 

0.59 

(0.41, 

0.87) 

0.61 

(0.52, 

0.71) 

0.67 

(0.46, 

0.97) 

0.71 

(0.64, 

0.80) 

0.82 

(0.74, 

0.90) 

0.82 

(0.68, 

0.98) 

0.83 

(0.75, 

0.93) 

0.88 

(0.68, 

1.13) 

0.90 

(0.40, 

2.05) 

0.91 

(0.05, 

14.92) 

0.95 

(0.34, 

2.71) 

1.30 

(1-08, 

1.57) 

1.34 

(0.99, 

1.80) 

1.42 

(1-09, 

1.86) 

0.72 

(0.61, 

0.86) 

0.54 

(0.22, 

1.31) 

0.67 

(0.12, 

3.88) 

0.68 

(0.62, 

0.76) 

0.78 

(0.40, 

1.49) 

0.99 

(0.36, 

2.70) 

1.15 

(0.92, 

1.43) 

1.20 

(0.02, 

75.01) 

1.30 

(1-09, 

1.55) 

0.90 

(0.65, 

1.27) 


.1 .25 .5 1 2 3 

Favours FFS participants Favours non-FFS 
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Figure A26 Pesticide use corrected for possible unit of analysis error: excluding high-risk- 
of-bias studies 


Study 

ID 

FFS participants 

Yamazaki & Resosudarmo, 2007 (In cfen e s i a) 
Yorobe et al., 201 1 (Philippines) 

Rejesus et al., 2010 (Vietnam) 

Pananurak, 2010 (Pakistan) 

Wu, 2010 (China) 

Praneetvatakul & Waibel, 2006 (Thailand) 

Ali & Sharif, 201 1 (Pakistan) 

Feder et al., 2004 (Indonesia) 

Cavatassi et al., 201 1 (Ecuador) 

Subtotal (l-squared = 83.0%, p = 0.000) 

FFS neighbours 

Yamazaki & Resosudarmo, 2007 (Indonesia) 
Wu, 2010 (China) 

Pananurak, 2010 (Pakistan) 

Praneetvatakul & Waibel, 2006 (Thailand) 
Feder et al., 2004 (Indonesia) 

Subtotal (l-squared = 91 .2%, p = 0.000) 



Response ratio 
ES (95% Cl) 


0.20 

(0.01, 

3.23) 

0.37 

(0.12, 

1.15) 

0.52 

(0.24, 

1.12) 

0.59 

(0.41, 

0.87) 

0.71 

(0.64, 

0.80) 

0.82 

(0.68, 

0.98) 

0.90 

(0.40, 

2.05) 

1.30 

(1.08, 

1.57) 

1.34 

(0.99, 

1.80) 

0.83 

(0.64, 

1.08) 


0.67 

(0.12, 

3.88) 

0.68 

(0.62, 

0.76) 

0.78 

(0.40, 

1.49) 

1.15 

(0.92, 

1.43) 

1.30 

(1.09, 

1.55) 

0.95 

(0.64, 

1.39) 


.1 .25 .5 1 2 3 

Favours FFS participants Favours non-FFS 



Figure A27 Other adoption measures corrected for possible unit of analysis error: all 
studies by risk of bias status 


Study 

ID 

High risk of bias 

Norvell & Hammig, 1999 (Indonesia) 
Friis-Hansen et at, 2004 (Uganda) 

Rao et at, 2012 (India) 

Endalew, 2009 (Ethiopia) 

Orozco Cirilo et at, 2008 (Mexico) 
Kelemework, 2005 (Ethiopia) 
Ricker-Gilbert et at, 2008 (Bangladesh) 
Amera, 2008 (Ethiopia) 

Mauceri et at, 2007 (Ecuador) 

Bunyatta et at, 2006 (Kenya) 

Dinpanah et at, 2010 (Iran) 

Subtotal (l-squared = 32.7%, p = 0.137) 

Medium risk of bias 

Todo & Takahashi, 201 1 (Ethiopia) 

Ali & Sharif, 2011 (Pakistan) 

Van Rijn, 2010 (Peru) 

Subtotal (l-squared = 79.7%, p = 0.007) 

Overall (l-squared = 50.1%, p = 0.017) 


Standardised mean difference 




❖ 

i 


T 


ES (95% Cl) 

0.08 

(-0.07, 0.23) 

0.10 

(-0.14, 0.33) 

0.11 

(-0.11, 0.33) 

0.24 

(-0.09, 0.58) 

0.64 

(-0.62, 1.90) 

0.70 

(-0.67, 2.06) 

0.92 

(0.29, 1.55) 

1.19 

(-1.14, 3.53) 

1.41 

(-1.35, 4.17) 

1.45 

(0.19, 2.70) 

1.46 

(-1.40, 4.31) 

0.22 

(0.05, 0.38) 

0.06 

(-0.06, 0.19) 

0.22 

(0.17, 0.27) 

0.46 

(0.23, 0.69) 

0.22 

(0.06, 0.38) 

0.21 

(0.11, 0.31) 


T 


-1 0 .5 1 2 5 

Favours non-FFS Favours FFS participants 


301 


Figure A28 Yields corrected for possible unit of analysis error: all studies 


Study 



Response ratio 

ID 



ES (95% Cl) 

IPM/IPPM FFS 




Pananurak, 2010 (India) 



0.80 (0.54, 1.20) 

Labarta, 2005 (Nicaragua) 

♦ 

0.97 (0.79, 1.18) 

Rejesus et al., 2010 (Vietnam) 



0.97 (0.72, 1.31) 

Feder et al., 2004 (Indonesia) 

4 

► 

0.98 (0.96, 1.01) 

Wu, 2010 (China) 


♦ 

1.08 (1.03, 1.14) 

Ali & Sharif, 201 1 (Pakistan) 



1.09 (0.56, 2.14) 

Yang et al., 2005 (China) 



1.15 (0.94, 1.41) 

Hiller et al., 2009 (Kenya) 



1.17 (0.53, 2.56) 

Davis et al., 2012 (Tanzania) 



1.23 (1.00, 1.51) 

Pananurak, 2010 (Pakistan) 



1.24 (1.01, 1.54) 

Wandji et al., 2007 (Cameroon) 


♦ 

1.32 (1.07, 1.63) 

Mutandwa & Mpangwa, 2004 (Zimbabwe) 


♦ 

1.36 (1.06, 1.73) 

Zuger, 2004 (Peru) 


♦ 

1.44 (1.09, 1.92) 

Carlberg et al., 2012 (Ghana) 



1 .58 (0.76, 3.29) 

Yamazaki & Resosudarmo, 2007 (Indonesia) 



1.67 (1.23, 2.26) 

Van den Berg et al., 2002 (Sri Lanka) 


♦ 

1.68 (1.30, 2.18) 

Davis et al., 2012 (Kenya) 



1.81 (1.15, 2.84) 

Subtotal (l-squared = 78.5%, p = 0.000) 


o 

1.20 (1.10, 1.30) 

Other FFS 




Van Rijn, 2010 (Peru) 



0.86 (0.63, 1.18) 

Naik et al., 2008 (India) 



0.89 (0.83, 0.96) 

Gockowski et al., 2010 (Ghana) 



1.14 (1.03, 1.25) 

Cavatassi et al., 201 1 (Ecuador) 



1.22 (0.97, 1.53) 

Birthal et al., 2000 (India) 



1 .24 (0.64, 2.39) 

Dinpanah et al., 2010 (Iran) 



1.32 (1.22, 1.42) 

Palis, 1998 (Philippines) 



1.36 (0.97, 1.92) 

Pande et al., 2009 (Nepal) 



2.11 (1.25, 3.56) 

Orozco Cirilo et al., 2008 (Mexico) 


♦ 

2.62 (2.23, 3.08) 

Todo & Takahashi, 201 1 (Ethiopia) 



> 2.71 (0.85, 8.66) 

Subtotal (l-squared = 94.7%, p = 0.000) 


o 

1.34 (1.07, 1.69) 

IPM neighbour farmers 




Pananurak, 2010 (India) 



0.79 (0.55, 1.14) 

Feder et al., 2004 (Indonesia) 

♦ 

0.99 (0.97, 1.01) 

Labarta, 2005 (Nicaragua) 

♦ 

1.00 (0.86, 1.17) 

Wu, 2010 (China) 



1.03 (0.99, 1.08) 

Pananurak, 2010 (Pakistan) 


1.03 (0.86, 1.25) 

Yamazaki & Resosudarmo, 2007 (Indonesia) 



1.43 (1.05, 1.96) 

Subtotal (l-squared = 51.5%, p = 0.067) 

< 

> 

1.01 (0.96, 1.07) 


T 


T 


.5 


2 3 6 


Favours non-FFS 


Favours FFS participants 
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Figure A29 Yields corrected for possible unit of analysis error: excluding high- risk- of- bias 
studies 


Study 

ID 

FFS participants 
Van Rijn, 2010 (Peru) 

Rejesus et at, 2010 (Vietnam) 

Feder et at, 2004 (Indonesia) 

Wu, 2010 (China) 

Ali & Sharif, 201 1 (Pakistan) 

Cavatassi et at, 20l'l (Ecuador) 

Davis et at, 2012 (Tanzania) 

Pananurak, 2010 (Pakistan) 

Yamazaki & Resosudarmo, 2007 (Indonesia) 
Davis et at, 2012 (Kenya) 

Todo & Takahashi, 201 1 (Ethiopia) 

Subtotal (l-squared = 81.1%, p = 0.000) 

FFS neighbours 

Feder et al., 2004 (Indonesia) 

Wu, 2010 (China) 

Pananurak, 2010 (Pakistan) 

Yamazaki & Resosudarmo, 2007 (Indonesia) 
Subtotal (l-squared = 65.9%, p = 0.032) 


o 


1 

.5 

Favours non-FFS 


Response ratio 

ES (95% Cl) 


0.86 ( 0 , 
0.97 (0, 
0.98 (0, 
1.08 (1 
1.09 (1 
1.22 ( 0 , 

1.23 (1 

1.24 (1 
1.67 (1 
1.81 (1 
2.71 (1 
1.13 (1 


.63, 1.18) 
.72, 1.31) 
.96, 1.01) 
.03, 1.14) 
.03, 1.15) 
.97, 1.53) 
.00, 1.51) 
.01, 1.54) 
.23, 2.26) 
.15, 2.84) 
.11,6.60) 
.04, 1 .22) 


0.99 (0.97, 1.01) 
1.03 (0.99, 1.08) 
1.03 (0.86, 1.25) 
1.43 (1.05, 1.96) 
1.02 (0.97, 1.08) 


1 1 r 

1 2 3 6 

Favours FFS participants 
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Figure A30 Net revenues corrected for possible unit of analysis error: all studies 


stud V Response ratio 

ID ES (95% Cl) 


IPM/IPPM FFS 

Labarta, 2005 (Nicaragua) < 

Pananurak, 2010 (India) 

Pananurak, 2010 (China) 

Pananurak, 2010 (Pakistan) 

Van den Berg et al., 2002 (Sri Lanka) 
Yang et al., 2005 (China) 

Khan et al., 2007 (Pakistan) 

Subtotal (l-squared = 33.9%, p = 0.169) 



0 


0.28 (0.02, 4.03) 
1.06 (0.60, 1.90) 
1.17(1.08, 1.27) 
1.23 (1.09, 1.40) 
1.41 (1.19, 1.67) 
1.53 (1.10, 2.15) 
3.40 (0.83, 13.86) 
1.26(1.14, 1.40) 


Other FFS curriculunh 
Waarts et al., 2012 (Kenya) 

Naik et al., 2008 (India) 

Van de Fliert, 2000 (Indonesia) 

Subtotal (l-squared = 0.0%, p = 0.598) 



1.14(0.92, 1.41) 
1.25(1.09, 1.42) 
1.31 (1.11, 1.55) 
1.24(1.14, 1.37) 


FFS+input/marketing 
Birthal et al., 2000 (India) 

Van Rijn, 2010 (Peru) 

Cavatassi et al., 201 1 (Ecuador) 

Palis, 1998 (Philippines) 

Subtotal (l-squared = 78.1%, p = 0.003) 



1.43 (0.68, 3.02) 
2.00(1.02, 3.94) 
3.34(1.56, 7.15) 
4.61 (3.83, 5.56) 
2.73 (1.50, 4.96) 


IPM neighbour farmers 

Pananurak, 2010 (India) » 

Pananurak, 2010 (China) ♦ 

Pananurak, 2010 (Pakistan) ♦ 

Labarta, 2005 (Nicaragua) r- 

Khan et al., 2007 (Pakistan) 

Subtotal (l-squared = 0.0%, p = 0.842) 0 


0.93 (0.58, 1.51) 
1.07(1.00, 1.14) 
1.13(1.01, 1.26) 
1.39 (0.57, 3.39) 
1.51 (0.19, 12.27) 
1.08(1.03, 1.15) 


.2 .5123 


Favours non-FFS 


Favours FFS participants 
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Figure A31 Net revenues corrected for possible unit of analysis error: excluding high-risk- 
of-bias studies 


Study 

ID 


Response ratio 
ES (95% Cl) 

FFS participants 



Pananurak, 2010 (China) 

« 

1.17 (1.08, 1.27) 

Pananurak, 2010 (Pakistan) 


1.23 (1.09, 1.40) 

Subtotal (l-squared = 0.0%, p = 0.461) 

0 

1.19 (1.11, 1.27) 

FFS+input/marketing 



Van Rijn, 2010 (Peru) 


— 2.00 (1.02,3.94) 

Cavatassi et at, 201 1 (Ecuador) 


• 3.34 (1.56, 7.15) 

Subtotal (l-squared = 0.0%, p = 0.327) 


> 2.51(1.51,4.16) 

FFS neighbours 



Pananurak, 2010 (China) 

n 

1.07 (1.00, 1.14) 

Pananurak, 2010 (Pakistan) 


1.13 (1.01, 1.26) 

Subtotal (l-squared = 0.0%, p = 0.424) 
1 1 

0 

1 r 

1.09 (1.03, 1.15) 


.2 .5123 

Favours non-FFS Favours FFS participants 
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META-ANALYSIS FINDINGS: ADDITIONAL ANALYSIS 


Figure A32 Sensitivity analysis: pesticide adoption for I PM/ 1 PPM FFS, excluding Feder et 
al. (2004) and Yamazaki and Resosudarmo (2008) 


Study 

ID 


FFS participants 

Birthal et at, 2000 (India) — 

Yorobe et at, 201 1 (Philippines) 

Yang et at, 2005 (Cnina) 

Khalid, n.d. (Sudan) - 

Rejesus et at, 201 0 (Vietnam) 

Pananurak, 2010 (India) — 

Mutandwa & Mpangwa, 2004 (Zimbabwe) 
Pananurak, 2010 (Pakistan) 

Amera, 2008 (Ethiopia) 

Mancini et at, 2008 (India) 

Wu, 2010 (China) 

Van den Berg et at, 2002 (Sri Lanka) 
Praneetvatakul & Waibel, 2006 (Thailand) 
Murphy et at, 2002 Vietnam) 

Cole et at, 2007 (Ecuador) 

AN & Sharif, 201 1 (Pakistan) 

Khan et at, 2007 (Pakistan) — 

Labarta, 2005 (Nicaragua) 

Cavatassi et at, 201 1 (Ecuador) 
Friis-Plansen et at, 2004 (Uganda) 
Subtotal (l-squared = 93.4%, p = 0.000) 



FFS neighbours 

Pananurak, 2010 (India) — 

Wu, 2010 (China) 

Pananurak, 2010 (Pakistan) 

Labarta, 2005 (Nicaragua) 

Praneetvatakul & Waibel, 2006 (Thailand) 
Khan et at, 2007 (Pakistan) 

Subtotal (l-squared = 73.7%, p = 0.002) 



Response ratio 
ES (95% Cl) 

0.21 ( 

;o.i7, 

0.26] 

0.37 ( 

0.18, 

0.78 

0.41 ( 

Jo. 36, 

0.46] 

0.48 ( 

;o.3i, 

0.75] 

0.52 | 

0.24, 

1.12 

0.52 < 

Jo.30, 

0.92) 

0.57 ( 

;0.36, 

0.89] 

0.59 < 

;o.4i, 

0.87; 

0.61 ( 

0.52, 

0.71 

0.67 ( 

0.46, 

0.97 

0.71 < 

[0.64, 

0.80] 

0.82 ( 

:o.74, 

0.90] 

0.82 ( 

0.68, 

0.98 

0.83 I 

0.75, 

0.93 

0.88 ( 

0.68, 

1.13 

0.90 I 

Jo.75, 

1.09; 

0.91 < 

[0.28, 

2.94] 

0.95 1 

Jo.39, 

2.34] 

1.34 < 

[0.99, 

1.80] 

1.42 ( 

1.09, 

1.86 

0.67 ( 

Jo.55, 

0.81] 

0.54 ( 

[0.25, 

1.15] 

0.68 ( 

Jo. 62, 

0.76] 

0.78 1 

;o.4o, 

1.49] 

0.99 ( 

Jo. 42, 

2.33] 

1.15 1 

[0.92, 

1.43] 

1.20 < 

[0.40, 

3.53] 

0.84 ( 

[0.61 , 

1.16] 


.1 .25 .5 1 2 3 

Favours FFS participants Favours non-FFS 
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Figure A33 Sensitivity analysis: pesticide adoption IP M/ IPPM FFS farmers by risk of bias 
status, excluding Feder et al. (2004) and Yamazaki and Resosudarmo (2008) 


Study 

ID 


Response ratio 
ES (95% Cl) 

High risk of bias 



Birthal et al., 2000 (India) 


0.21 (0.17, 0.26) 

Yang et al., 2005 (China) B 


0.41 (0.36, 0.46) 

Khalid, n.d. (Sudan) — ♦ 


0.48 (0.31, 0.75) 

Pananurak, 2010 (India) • 


0.52 (0.30, 0.92) 

Mutandwa & Mpangwa, 2004 (Zimbabwe) • 


0.57 (0.36, 0.89) 

Amera, 2008 (Ethiopia) -B 

r 

0.61 (0.52, 0.71) 

Mancini et al., 2008 (India) — ^ \ 


0.67 (0.46, 0.97) 

Van den Berg et aU 2002 (Sri Lanka) 

I -# 

0.82 (0.74, 0.90) 

Murphy et al., 2002 Vietnam) 

■# 

0.83 (0.75, 0.93) 

Cole et al., 2007 (Ecuador) 

i 

i * 

0.88(0.68,1.13) 

Khan et al., 2007 (Pakistan) 

i t 

\ ™ 

0.91 (0.28, 2.94) 

Labarta, 2005 (Nicaragua) 


0.95 (0.39, 2.34) 

Friis-Hansen et al., 2004 (Uganda) 

i 

— H- 1.42(1.09,1.86) 

Subtotal (l-squared = 95.1%, p = 0.000) <C> 

i 

0.63 (0.48, 0.83) 

i 

Medium risk of bias 

i 

i 


Yorobe et al., 201 1 (Philippines) ♦ i 

1— 

0.37 (0.18, 0.78) 

Rejesus et al., 201 0 (Vietnam) ♦ 

1 

i 

0.52(0.24,1.12) 

Pananurak, 2010 (Pakistan) ■» 

i 

i 

0.59 (0.41, 0.87) 

Wu, 2010 (China) 

0.71 (0.64, 0.80) 

Praneetvatakul & Waibel, 2006 (Thailand) 

i— ♦— 

0.82 (0.68, 0.98) 

Ali & Sharif, 201 1 (Pakistan) 

i 0.90 (0.75, 1.09) 

Cavatassi et al., 201 1 (Ecuador) 

i 

i 

— •— 1.34(0.99,1.80) 

Subtotal (l-squared = 75.6%, p = 0.000) ^C> 

0.78 (0.64, 0.96) 

Overall (l-squared = 93.4%, p = 0.000) <^> 

T T 1 — ^ 

0.67 (0.55, 0.81) 
1 T 


.1 .25 .5 1 2 3 

Favours FFS participants Favours non-FFS 
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Figure A34 Sensitivity analysis: yields for FFS farmers, excluding Feder et al. (2004) and 
Yamazaki and Resosudarmo (2008) 


Study 

ID 


Response ratio 

ES (95% Cl) 


FFS participants 
Pananurak, 2010 (India) 


0.80(0.61,1.05) 

Van Rijn, 2010 (Peru) 


— 0.86(0.63,1.18) 

Naik et al., 2008 (India) 


0.89 (0.83, 0.96) 

Labarta, 2005 (Nicaragua) 

♦ 

0.97 (0.92, 1.02) 

Rejesus et al., 201 0 (Vietnam) 


0.97(0.72,1.31) 


Wu, 2010 (China) 

AN & Sharif, 201 1 (Pakistan) 

Gockowski et al., 2010 (Ghana) 

Yang et al., 2005 (China) 

Hiller et al., 2009 (Kenya) 

Khan et al., 2007 (Pakistan) 

Cavatassi et al., 201 1 (Ecuador) 

Davis et al., 2012 (Tanzania) 

Birthal et al., 2000 (India) 
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Figure A35 Sensitivity analysis: yields for FFS farmers by risk of bias status, excluding 
Feder et al. (2004) and Yamazaki and Resosudarmo (2008) 
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Figure A36 Disaggregated IPM practices adopted by FFS farmers (Ricker- Gilbert et al.. 


2008, Bangladesh) 
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Figure A37 Disaggregated IPM practices known by FFS neighbours (Ricker- Gilbert et al., 
2008, Bangladesh) 
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Appendix H 


DESCRIPTIVE SYNTHESIS OF FINDINGS FROM 
QUALITATIVE STUDIES 


From the line by line coding of the primary studies we extracted descriptive themes 
(following Thomas & Harden, 2008), which remained dose to the findings in the 
primary studies. We summarised relevant findings from each study and synthesised 
emerging themes into statements and concepts. This descriptive synthesis provided 
the basis for the causal chain synthesis and is reported below. The statements in 
bold represent different descriptive themes identified based on the findings of the 
induded studies. 

Targeting and participation 

Targeting, partidpant sdection and group composition 

An important factor for FFS effectiveness is that appropriate farmer groups are 
targeted and that partidpants are motivated and able to attend field schools 
throughout the full season. While studies provide information on targeting, and 
selection procedures and group composition, their link to FFS effectiveness is not 
well explored. Twelve studies out of twenty report some information on targeting, 
selection and composition of FFS groups (DANIDA, 2011; Friis- Hansen, 2008; 
Gottret & Cordoba, 2004; Hofisi, 2003; Machacha, 2008; Najjar, 2009; Palis, 2002; 
Pedersen et al., 2008; Rola &Baril, 1997; Simpson, 1997; Van de Fliert, 1993; Van 
Der Wiele, 2004). Six studies report on factors affecting partidpation and drop-out 
rates. Only four studies directly or indirectly discuss the implications of targeting 
and group composition on FFS effectiveness (Gottret & Cordoba, 2004; Hofisi, 
2003; Machacha, 2008; Pedersen et al., 2008). 

Targeting procedures appear to vary from context to context. In some instances, 
farmers were selected according to predetermined criteria (DANIDA, 
2011; Palis, 2002; Simpson, 1997). In the Philippines, partidpants had to be active 
farmers, able to attend all FFS sessions, and able and willing to share what they 
learned with other farmers (Palis, 2002). The chairman responsible for sdecting 
partidpants also tried to ensure geographic representation among partidpants. 
Similarly, in Cambodia, partidpants had to be active rice farmers, willing to learn 
and willing to share what they learned with other farmers ( Simpson, 1997) . 
Partidpation of women was encouraged. 
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One study mentions that participants were targeted through established 
contact people associated with the implementing agency (Van Der Wiele, 2004). 
The study notes that less than half of the interviewed participants 75 had farming as 
their main occupation, suggesting that this targeting approach may not have been 
effective. 

In Bangladesh, Indonesia and Cambodia the elite and more affluent members 
of the community were selected for participation in the FFS (DAM DA, 
2011; Simpson, 1997; Van de Fliert, 1993). In Cambodia, participants were selected 
by village leaders who were privileging members of the community that were well 
connected with the local leader or the NGO. The selection criteria "implicitly 
excluded poor farmers, especially widows as criteria included literacy, numeracy and 
social standing 7 ', thereby excluding more than one- fifth of the community (Simpson, 
1997, pp. 146- 147). In Bangladesh, the selection procedures excluded the landless 
and share-croppers (DAMDA, 2011). In Indonesia, FFS groups were composed of 
the more affluent and more informed farmers. The author suggests that this is the 
result of selection procedures being "strongly based on customaiy social patterns in 
J avanese society" (Van de Fliert, 1993, p. 227). 

Three studies found that targeting procedures led to some women and the 
poor being excluded from participating in FFS (DAMDA, 2011; Najjar, 

2009; Simpson, 1997) . In Cambodia, many women and poorer families could not 
participate because farmers were required to be literate in order to participate and 
widows were often overlooked during selection procedures (Simpson, 1997) . 
Similarly, in Bangladesh, the selection procedures excluded women- headed 
households, widows, the landless and share-croppers (DAMDA, 2011). In Kenya, 
while the majority of participants were women, female- headed households, divorced 
and single mothers faced prohibitive recruitment procedures, such as selection by 
local administration or recruitment in public barazas (Najjar, 2009). 

In many instances, farmers self- selected into FFS groups, chose not to participate or 
were prevented from participating by various factors unrelated to targeting. Eleven 
studies explored the motivating factors for self- selection and reasons for non- 
participation in FFS (DAMDA, 2011; Friis- Hansen, 2008; Friis- Hansen et al., 2012; 
Gottret& Cordoba, 2004; Hofisi, 2003; Machacha, 2008; Najjar, 2009; Rola& 

Baril, 1997; Van de Fliert, 1993; Van Der Wiele, 2004). 

In some instances, participants chose to attend FFS based on their interest 
in learning (Friis- Hansen, 2008; Hofisi, 2003; Najjar, 2009). In Zimbabwe, elite 
farmers did not think they would benefit from participation in FFS and perceived 
farmer learning as a waste of time. However, the study also mentions that some 
innovative elite farmers did not share these views and took pride in participating in 
the FFS (Hofisi, 2003) . In Kenya, non- participating farmers mentioned lack of 
interest or their attachment to previously held knowledge as reasons for non- 


75 The authors interviewed around two- thirds of all FFS participants. 
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participation. The study also notes lack of interest as one of the reasons for non- 
participation by the youth (Najjar, 2009). In Uganda, the implementing agency 
conducted a sensitisation around a common interest in learning new skills prior to 
group formation (Friis- Hansen, 2008, p. 519). The sensitisation was not entirely 
successful but farmers not interested in learning soon left the groups as they did not 
consider the time investment worth the benefits. Those that remained were those 
who shared willingness to "invest time and effort in learning and in conducting joint 
activities" (Friis- Hansen, 2008, p. 519). 

Those attending FFS reported that improving livelihoods was a motivating 
factor for joining FFS (DANIDA, 2011; Friis- Hansen et al., 2012; Najjar, 2009). 
In one FFS in Kenya, both men and women were seeking to gain new skills, increase 
food production and increase income (Najjar, 2009). Friis- Hansen and colleagues 
(2012) report similar findings from their study in Kenya. They note that "frustration 
over their livelihoods and aspirations for a better fife ultimately became the key 
motivators forjoiningFFS" (Duveskoget al., 2011, p. 1,535). In Bangladesh, the 
motivating factor was the acquisition of knowledge that would improve farmers' 
agricultural production and incomes (DANIDA, 2011). Two studies note poverty 
and poor health as reasons for non-participation (Gottret & Cordoba, 2004; 
Najjar, 2009). A study notes that many poor farmers in Honduras are unable to 
participate unless they are to be paid for their time spent in FFS (Gottret & Cordoba, 
2004) . In Kenya, the youth lacked access to land, but also did not participate due to 
poor health (Najjar, 2009) . A study from Honduras reported that more educated 
farmers with positions of responsibility were more likely to participate (Gottret & 
Cordoba, 2004). 

Five studies found that women were often not able or allowed to participate 

(DANIDA, 2011; Hofisi, 2003; Najjar, 2009; Van deFliert, 1993; Van DerWiele, 
2004). In Indonesia, no women were observed to participate in FFS (Van de Fliert, 
1993) and in Liberia women participants tended to be in the minority despite 
intentions for equal representation of genders (Van DerWiele, 2004). In 
Bangladesh, the wives of men who did not have time to attend FFS were not allowed 
to take their husbands' place due to the presence of other men in the group 
(DANIDA, 2011). 

Two other studies report that some women were not allowed to participate by their 
husbands (Hofisi, 2003; Van DerWiele, 2004). In Liberia and Kenya, some women, 
especially those heading households, divorced and single mothers, did not have time 
to participate due to child- care requirements and other household responsibilities 
(Najjar, 2009; Van DerWiele, 2004). Other barriers to women's participation 
included lack of access to land (Van DerWiele, 2004; Hofisi, 2003), appropriate 
tools and lack of opportunities to practise their FFS skills in the fields as husbands 
considered this a waste of time (Van Der Wiele, 2004) . The women farmers noted 
that "there is need to work with the local leadership such as chiefs and headmen on 
FFS so that women farmers (who are the majority of those who attend schools) are 
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given land from where to practise if they are not given land by their husbands" 
(Hofisi, 2003, p. 42). A study in Kenya likewise found that participation of female- 
headed households, divorced and single mothers was limited due to stigma (Najjar, 
2009). 

Other studies report on FFS where women were well represented in the FFS 

groups (Hofisi, 2003; Machacha, 2008; Najjar, 2009; Rola&Baril, 1997). In some 
cases this was because the men could not attend the FFS due to off-farm work 
commitments and seasonal out-migration (Najjar, 2009; Hofisi, 2003). However, in 
Kenya, multiple key informants noted that while men may not attend field schools, 
they remain the decision-makers on farms (Machacha, 2008, p. 9), implying that 
women FFS participants may not always be able to apply their FFS knowledge and 
skills in the fields. A study from Kenya found that men often did not participate 
because they did not want to be involved on an egual standing with women or did 
not want to be in groups led by women (Najjar, 2009) . Although the study also 
mentions that the curriculum failed to attract male participants because it did not of 
focus on the issues of most concern to men, including commercial farming and 
livestock (Najjar, 2009). 

Targeting, participation and FFS effectiveness 

Four studies directly or indirectly discuss finks between targeting and FFS 
effectiveness (Gottret &Cordoba, 2004; Hofisi, 2003; Machacha, 2008; Pedersen et 
al., 2008). Pedersen and colleagues (2008) suggest that lack of appropriate 
targeting might be a barrier to successful FFS. The authors found that the 
selection of participants in a Tanzanian FFS was not refined enough, with all 
participants being admitted regardless of whether they were suitable or not. The 
authors suggest that this may have contributed to lack of knowledge and skills 
formation, yet they do not provide additional information to substantiate this claim. 

Two studies suggest that distance to FFS may be an important factor for 
effective FFS functioning (Hofisi, 2003; Machacha, 2008). A study from Uganda 
suggests that large distances between members' homes were an important factor in 
FFS performance. Farmer facilitators had trouble reaching the remote- living 
farmers and the author suggests that this has contributed to the disintegration of the 
group (Hofisi, 2003) . A study from Kenya suggests that the most successful FFS was 
also the one whose members lived closest together (Machacha, 2008) . 

Finally, two studies suggest that participant characteristics may affect the 
extent to which they can benefit from FFS (Gottret & Cordoba, 2004; Hofisi, 
2003) . These refer particularly to education levels and the skills they developed 
through informal learning (Gottret & Cordoba, 2004) . A study from Zimbabwe 
likewise reports that FFS with good attendance and confident and motivated 
farmers performed better than FFS with poor attendance and shy farmers (Hofisi, 
2003). 
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Participant drop-out 

Participant drop-out is another factor that is likely to affect FFS effectiveness. Six 
studies discuss participant drop-out. Five studies report high levels of drop-out 
(Friis- Hansen, 2008; Gottet &Cordoba, 2004; Machacha, 2008; Najjar, 2009; Rola 
&Baril, 1997; Winarto, 2004). In Indonesia, Winarto (2004) reports on having to 
abandon field work in one of the hamlets due to the high level of drop-outs, noting 
that the FFS would not reach its objectives as participants were replaced by group 
members who were not farmers (Winarto, 2004, p. 34). The study does not provide 
reasons for this high drop-out rate. 

Studies that do report reasons for drop-out suggest that drop-out was due to 
unmet expectations of hand-outs or loan availability (Friis- Hansen, 2008; 
Hofisi, 2003; Machacha, 2008; Najjar, 2009). In Zimbabwe, farmersjoineddueto 
seed loans and ended up leaving when the loans were stopped. The study notes that 
participants replacing drop-outs found it difficult to catch up on previously covered 
material (Hofisi, 2003) . A study in Uganda reporting high levels of drop-out (with 
up to half of the FFS group members leaving within the first year), notes that despite 
a sensitisation process, some fanners "still joined FFS groups primarily because of 
an interest in accessing external funds" (Friis- Hansen, 2008, p. 519). Two studies 
from Kenya report that many farmers dropped out or refrained from participating 
when realising that cash was not forthcoming (Machacha, 2008; Najjar, 2009) . 

One of the studies also mentions increases in individual commitments as 
factors leading to participants dropping out (Machacha, 2008). The study 
suggests that absences and drop-outs are a barrier to effective FFS functioning, as 
there are not enough members to care for the communal plot. The study refers to an 
FFS targeting youth and notes that "the fact that not all the members are always 
present due to commitments at school or work, and that those starting new careers 
or families move away, have contributed to operational difficulties for the group" 
(Machacha, 2008, p. 38). 

Additional reasons for drop-outs included farmers not being interested in the 
curriculum and limited access to transportation (Gottet & Cordoba, 2004). 


FFS facilitators 

Facilitator selection and characteristics 

The facilitators play a key role in the FFS, and ensuring that this role is performed by 
suitable candidates may influence the success of the FFS. Four studies included 
themes pertaining to facilitator selection and characteristics (DANIDA, 2011; Hofisi, 
2003; Machacha, 2008; Van Der Wide, 2004) 

Three studies suggest appropriate criteria for selecting facilitators are 
important for identifying suitable candidates, with personal attitude and 
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leadership skills appearing important (DAMDA, 2011; Hofisi, 2003; 
Machacha, 2008). Hofisi (2003) attributes the poor performance of some groups to 
poorly performing facilitators, and suggests an important reason for this was that 
many of the FFS groups adopted inadequate criteria when selecting their facilitators, 
focusing on high level of education, rather than attitude, maturity, literacy and 
farming experience. Consequently, some of the selected facilitators were not suitable 
and poorly performing. In Bangladesh the personal facilitation, organisational and 
leadership skills were found to be important criteria for selecting good facilitators 
(DAMDA, 20 11) . Similarly, a study from Kenya found that good leadership, rather 
than the education level of the facilitator was important for FFS performance 
(Machacha, 2008). 

Two studies suggest that women preferred female facilitators (Hofisi, 2003; 
Van Der Wiele, 2004) . In Zimbabwe women reported being more comfortable 
working with a female facilitator (Hofisi, 2003). In Liberia, women reported being 
comfortable training with men, but also preferred having a female facilitator as they 
considered she would have a greater understanding of their problems and 
perspectives (Van Der Wiele, 2004). 

FFS facilitator training and performance 

Adequate training of facilitators is a key assumption for FFS effectiveness. If 
facilitators are not adequately trained this has serious implications for the quality of 
the training received by farmers, affecting knowledge formation, adoption and final 
outcomes. Mne of the included studies report on themes relating to the training and 
performance of FFS facilitators (DAMDA, 2011; Gottret & Cordoba, 2004; Hofisi, 
2003; Najjar, 2009; Pedersen et al., 2008; Rola&Baril, 1997; Simpson, 1997; Van 
deFlierf, 1993; Winarto, 2004). 

One of the studies from Indonesia (Van de Fherf, 1993) found that facilitators 
received training, were knowledgeable and had good facilitation skills. 

The facilitators had previously performed the role as pest observers, and received 
training over a 15- month period prior to taking up their positions as FFS facilitators. 
The facilitators'/ pest observers' facilitation skills and practical knowledge brought 
acceptance among farmers, while the performance of the extension workers varied 
more, depending on their interest in I PM and relationship with the pest observer. 

Another study from Indonesia (Winarto, 2004), suggests that farmers' views of the 
facilitators were more lukewarm. It highlights that facilitators did not have 
previous experience of farming, nor extension before taking on their role as 
FFS facilitators. Prior to entering the training to become facilitators the pest 
observers focused on monitoring pest populations and developing control strategies 
that relied heavily on the use of pesticides. Some of the IPM farmers also expressed 
dissatisfaction with trainers not practising what they preached - the facilitators did 
not implement the practices together with the farmers, and did not cultivate rice 
themselves, to allow for farmers to observe their outcomes and be more confident in 
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the messages. 


Six studies report issues related to a lack of appropriate training, resources 
and ongoing support of facilitators (DANIDA, 2011; Gottret & Cordoba, 2004; 
Hofisi, 2003; Pedersen et al., 2008; Rola &Baril, 1997; Simpson, 1997). While 
extensionists in Tanzania were provided with a season-long FFS training- of- trainers 
course (Pedersen et al., 2008), the study found that the facilitators lacked the skills 
and dynamics to deliver high-quality FFS. The reason for this was a combination of 
lack of sufficient initial training, dear terms of reference, infrequent field 
monitoring and backstopping from the district, and lack of inputs (e.g. meals, fuels, 
allowances) for fadlitators to run FFS. 

In Nicaragua, the training provided to the fadlitators did not cover marketing and 
commercialisation, despite this being part of the FFS curriculum (Gottret & 
Cordoba, 2004) . While the facilitators recognised the importance of these topics, 
they lacked the tools and methodologies needed to fadlitate sessions on these topics. 
In Bangladesh and Cambodia the curriculum for the training of trainers was found 
to be too technical, with insuffident focus on developing partidpatory fadlitation 
skills (DANIDA, 2011; Simpson, 1997). 

Two studies, from the Philippines and Zimbabwe, highlight that suffident finandal 
incentives were lacking (Hofisi, 2003; Rola & Baril, 1997) . This left fadlitators 
demoralised, with some not spending suffident time at the field school and farm 
visits. This in turn demoralised the farmers, who felt unable to hold the fadlitator to 
account as he/ she was not answerable to the farmers. 

Relationship between farmers and fadlitators 

Finally, five studies highlight themes relating to the relationship between farmers 
and facilitators (Isubikalu, 2007; Machacha, 2008; Najjar, 2009; Simpson, 1997; 
Van de Fliert, 1993) . Three of the studies suggest an imbalance in the farmer- 
facilitator relationship as a potential barrier to farmers learning and 
adoption (Isubikalu, 2007; Najjar, 2009; Simpson, 1997). Two studies from 
Uganda and Kenya suggest that in many cases the fadlitators failed to treat farmers 
as co- learners, but rather as students (Isubikalu, 2007; Najjar, 2009). In Kenya, 
they dictated notes and in some cases imposed a crop to be cultivated in the fidd 
school, with negative implications for farmers' learning. Nevertheless, the study also 
reports instances of fadlitators being more open for farmers' partidpation in 
discussions and actively solidtingpartidpants' opinions (Najjar, 2009). 

Similarly, in Cambodia the fadlitators preferred a more formal approach to the FFS 
training, retaining control of the sessions. Farmers on the other hand did not feel 
comfortable with voidng concerns or making suggestions to the fadlitators because 
the fadlitators were from urban and educated backgrounds and "must therefore be 
respected ... He knows many things so we listened to him. Sometimes I didn't fully 
understand him but I did not interrupt" (Simpson, 1997, p. 141). 
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In Indonesia Van de Fliert ( 1993) found that while the pest observers succeeded in 
facilitating, rather than teaching in the fanner field schools, they struggled to 
balance facilitation with leadership. The study found that the facilitators at 
times struggled with providing leadership to groups and performing tasks such as 
managing the time, calling off discussions that were too long and reigning in 
participants that were too dominant in order to provide room for others to 
participate. The author suggests that this might be due to facilitators' fears of being 
too dominant and suggests more training in group leadership is needed to improve 
the facilitation skills of the facilitators. 

Drawing on four case studies, a study in Kenya suggests that the rapport between 
facilitators and farmers may influence FFS effectiveness (Machacha, 

2008) . In three of the FFS in the study the rapport between the farmers and their 
facilitators was reported to be excellent, aiding the success of the group, while in one 
group the relationship between the members and the facilitator was found to be out 
of harmony, as evidenced by low attendance rates by both farmers and facilitators. 

Discourse and language of instruction used in farmer field schools 

Four of the studies included themes pertaining to the discourse and language of 
instruction adopted in the farmer field schools (Dolly, 2009; Najjar, 2009; Van de 
Fliert, 1993; Winarto, 2004). 

Two of the studies, both from Indonesia, found that the use of alien and 
complex concepts was a barrier to knowledge formation and adoption 

(Van de Fliert, 1993; Winarto, 2004). Facilitators used new foreign and scientific 
terms such as "economic threshold level" (ETL) and "ecosystem", which were part of 
their vocabulary, but these terms were not easily understood by the farmers. While 
such terms were to a certain extent incorporated into the language of I PM farmers, 
their use of these terms was more limited and the concepts were interpreted in 
different ways. Other terms, such as "natural enemy" (musuh alami) or "farmers' 
friends" (teman petani) were more easily understood. 

On the other hand, the use of common concepts and metaphors facilitated 
knowledge formation and adoption (Dolly, 2009; Winarto, 2004). The use of 
analogies and metaphors was an important part of the farmers' learning process. In 
the paradigm that preceded IPM in Indonesia the use of the analogy of spraying 
crops with "medicine" to combat pests had been prevalent, and the use of metaphors 
such as calling pesticides "poison", predators "farmers' friends and helpers" (p. 347) 
and "understanding the pest's behaviour" (p. 358) were found to be important in 
enhancing farmers' understanding of the scientific knowledge which forms the basis 
of IPM and changing the paradigm regarding pesticide use. In Trinidad and Tobago, 
the farmers, scientists and extension officers developed a common vocabulary over 
time (Dolly, 2009). 

In Bangladesh, Kenya and Indonesia knowledge acquisition was influenced 
by the language of instruction (DANIDA, 2011; Najjar, 2009; Van de Fliert, 


318 



1993) . In Kenya the use of Swahili instead of relevant local languages as the 
language of instruction was a barrier for women's participation and learning (Najjar, 
2009) . In Bangladesh the use of Bangla instead of local languages was found to be 
inappropriate, suggesting important messages may have been lost in translation, 
and hampered by cultural and religious differences between the participants and 
facilitators (DANIDA, 2011). On the other hand, in Indonesia (Van de Fliert, 1993) 
farmers' understanding was found to be enhanced when trainers used the local 
language instead of Indonesian, the national language. The assumption that farmers 
were sufficiently fluent in their national language to be able to comprehend the 
content of the FFS appeared to be incorrect. 


Mode of delivering FFS training 

The theory of FFS stresses the participatory, discovery- based learning underlying 
learning in the field schools and this is considered one of the main differences 
between FFS and more traditional extension approaches. Few of the themes 
identified in the descriptive synthesis are repeated in more than a handful of studies, 
but well over half of the included studies included themes pertaining to the way in 
which FFS were delivered (DANIDA, 2011; Dolly, 2009; Friis- Hansen, 2008; Gottret 
&C6rrioba, 2004; Hofisi, 2003; Isubikalu, 2009; Machacha, 2008; Najjar, 2009; 
Pedersen et al., 2008; Rola &Baril, 1997; Van de Fliert, 1993; Winarto, 2004). 

Four studies from Cambodia, Uganda, Tanzania and Kenya suggest that overall FFS 
were delivered in a top-down manner, using a transfer of technologies 
approach rather than in a participatory, discovery- based manner (Isubikalu, 2009; 
Pedersen et al., 2008; Najjar, 2009; Simpson, 1997). For instance, in Uganda 
farmers' participation in agro- ecosystem analysis (AESA) was limited to collecting 
data, with facilitators managing the experiments (Isubikalu, 2009) . There was a bias 
towards talk, rather than practice, and the author suggests that "this converts 
extension practice back into the model it displaced - Training and Visit". In 
Tanzania the programme followed a participatory approach for group formation, but 
the overall delivery of the FFS was characterised by a top-down, or "guided- choice" 
approach to budgeting, monitoring and provision of inputs provided (Pedersen et 
al., 2008). Similarly, in Kenya the FFS programme was mainly focused on 
transferring technologies and the author suggests that a lack of adaptation of the 
intervention to the Kenyan context might be a reason for the lack of local innovation 
(Najjar, 2009). There were also reports of coercion affecting the learning conditions. 
Finally, in Cambodia there was a lack of consultation during both the planning and 
implementation of the intervention. The author suggests that the decision to 
implement IPM-FFS was made by international aid organisations. While the farmers 
reported to enjoy the practical and non- formal aspects of the FFS, the study suggests 
that the experiments were led by the facilitators and that the farmers did not feel 
empowered to conduct their own experiments (Simpson, 1997). 
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The descriptive themes from the findings in the other studies are more positive. 

Nine studies from Bangladesh, Trinidad and Tobago, Uganda, Kenya, Honduras, 
Zimbabwe, the Philippines and Indonesia suggest that FFS were delivered in a 
participatory, bottom-up manner (DANIDA, 2011; Dolly, 2009; Friis- Hansen 
etal., 2008; Gottret & Cordoba, 2004; Hofisi, 2003; Machacha, 2008; Rola&Baril, 
1997; Van de Fliert, 1993; Winarto, 2004). 76 The training was mainly delivered 
through facilitation following experiential learning and empowerment approaches 
rather than lecturing, and focused on principles rather than predetermined 
recommendations. For instance, in Zimbabwe Hofisi (2003) found that the FFS was 
implemented in a participatory manner, with farmers being involved in 
experimentation and development and adaptations of the curriculum. Farmers were 
encouraged to experiment and the innovations resulting from farmers' experiments 
were taken seriously. According to the participating farmers, the active 
experimentation and information sharing enhanced their learning and increased 
knowledge and ownership of the resulting farming systems (Hofisi, 2003) . 

Nevertheless, one of the studies from Indonesia (Winarto, 2004) also found that 
trainers did at times slip back to the "chalk and talk" methods typically used in 
extension. In Honduras the FFS training was found to be delivered in a participatory 
manner, although the community sometimes agreed to cover crops they were not 
interested in, in order to stay on good terms with the community leaders (Gottret & 
Cordoba, 2004) . Similarly, in Kenya the training sessions were found to be 
participatory, and while there was farmer experimentation and some involvement in 
curriculum design, the facilitating organisation was the main decision-maker 
(Machacha, 2008). 


FFS content and coverage 

The relevance of the FFS curriculum to farmers' needs and appropriateness of new 
practices to the local context are important assumptions of successful FFS. Thirteen 
of the studies included themes relating to the coverage and content of the 
curriculum (DANIDA, 2011; David, 2007; Dolly, 2009; Gottret & Cordoba, 2004; 
Hiller etal., 2009; Hofisi, 2003; Isubikalu, 2007; Mandni etal., 2007; Najjar, 2009; 
Palis, 2002; Simpson, 1997; Van de Fliert, 1993; VanDerWiele, 2004). 

Appropriateness of the FFS content and coverage 

Eleven of the studies that included themes relating to the content and coverage of 
FFS report findings pertaining to the relevance and appropriateness of FFS to the 
local context (DANIDA, 2011; Dolly, 2009; Gottret & Cordoba, 2004; Hiller et al., 
2009; Hofisi, 2003; Isubikalu, 2007; Mandni etal., 2007; Najjar, 2009; Simpson, 
1997; Van de Fliert, 1993; Van Der Wiele, 2004). 


76 The description and analysis of actual implementation of FFS is limited in both Friis- Hansen (2008) 
and Friis- Hansen et al. (20 12) . They provide what is a rather generic description of an "ideal type” FFS, 
backed up with very limited description and no data on implementation. 
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Of these, seven studies of FFS implemented in Bangladesh, Trinidad and Tobago, 
Nicaragua, Kenya, Zimbabwe, India and Indonesia suggest that the curriculum 
was relevant and appropriate to the local context, facilitating knowledge 
acguisition and adoption (DANIDA, 2011; Dolly, 2009; Gottret & Cordoba, 2004; 
Hiller et al., 2009; Hofisi, 2003; Mandni et al., 2007; Van de Fliert, 1993). 

For instance, in India and Trinidad and Tobago the FFS curriculum responded to 
local concerns over the economic and environmental costs of pesticides respectively 
(Dolly, 2009; Mandni etal., 2007). In Bangladesh the majority of technologies 
induded in the FFS curriculum seemed relevant, although some curriculum 
components included technology which was too expensive or inappropriate, and 
therefore considered irrelevant (DANIDA, 2011). While the curriculum was 
considered relevant by farmers in Honduras, a greater focus on cash crops would 
have been desirable (Gottret & Cordoba, 2004) . 

Partidpants in Zimbabwe in general found that the topics covered in the FFS were 
applicable to their situation (Hofisi, 2003) .The relevance of each topic depended on 
the problems different farmers faced, and the farmers would focus on the solutions 
they found most relevant to their problems. The inclusion of indigenous 
knowledge increased the sense of ownership and motivated farmers' 
learning, as well as contributing to the relevance of the curriculum (Hofisi, 2003) . 

On the other hand, three studies induded themes relating to a lack of relevance 
and appropriateness of the curriculum to the local context (Isubikalu, 
2007; Najjar, 2009; Simpson, 1997). In Kenya the information provided regarding 
soil fertility and pest management was only appropriate and accurate for farmers 
from the high potential areas (Najjar, 2009) . The focus of the FFS was intended to 
be on diversification of food production, yet the crop which most FFS focused on 
was maize (this crop was also inappropriate for most of the area) . 

In Uganda, where pesticide use is not a major problem, IPM was found to be less 
appropriate (Isubikalu, 2007). The major technical component of FFS was improved 
crop varieties, but these lacked the features valued by most subsistence farmers such 
as good taste. The study also suggests that the practices proposed in most FFS were 
too labour- and time- intensive to be affordable for most farmers, and suggest that 
they may be more appropriate for commercially oriented farmers. 

In Cambodia the IPM programme focused on environmental problems, but farmers 
were more concerned about soil fertility and income generation (Simpson, 1997). 
One of the intended benefits of the IPM training was a reduction in farmers' 
spending on pesticides, but this was already low among the partidpants. The most 
successful experiment was the use of cow dung to fertilise crops, but the level of cow 
dung required was well beyond what farmers could access, and farmers were 
therefore unable to replicate this practice in their fields. 

In Liberia practices that reduced farmers' vulnerability were relevant and 
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appropriate (Van Der Wiele, 2004) . The FFS in Liberia aimed at reducing fanners' 
vulnerability in a context of chronic conflict and instability, but some of the 
practices, such as cultivation of vegetable crops or beekeeping, required significant 
commitment of time and financial resources and were perceived to increase 
vulnerabihty. On the other hand, practices which reduced farmers' reliance on 
purchased inputs reduced vulnerabihty and were more readily adopted. 

Comprehensiveness of FFS content and coverage 

Six of the studies that included themes relating to the content and coverage of FFS 
report findings pertaining to the comprehensiveness of FFS content and coverage 
(DANIDA, 2011; Mandni et al., 2007; Najjar, 2009; Pedersen et al., 2008; Van de 
FLiert, 1993; Winarto, 2004). Five of these studies suggest that the FFS curriculum 
was not sufficiently comprehensive in its coverage (DANIDA, 2011; Mandni et 
al., 2007; Najjar, 2009; Van de Fliert, 1993; Winarto, 2004). While the focus of the 
FFS might be relevant and appropriate, farmers often have a range of concerns. 
Failure to incorporate a broader range of concerns in the curriculum was found to be 
a major weakness in several studies. For instance, in Kenya, the lack of focus on 
water, irrigation, marketing and sodetal issues impeding agricultural production 
was found to be a major weakness (Najjar, 2009). Similarly, in India farmers had a 
range of concerns, such as water management, crop rotation, crop diversification 
and marketing. Discussions with farmers suggest that focusing on more than one 
crop, and adopting a broader systems approach might have improved the 
effectiveness of the IPM-FFS (Mandni et al., 2007). In Indonesia it was found that 
the indusion of other issues, such as fertilisation, in the FFS curriculum would also 
have been relevant for partidpating farmers (Van de Fliert, 1993) . The farmers did 
not gain suffident knowledge and skills to be able to deal with the range of complex 
issues they face in their fields, including how to deal with a new, unfamiliar pest 
(Winarto, 2004; Van de Fliert, 1993). 

Two studies found the curriculum was too broad to cover the selected 
topics comprehensively (Hofisi, 2003; Pedersen et al., 2008). The FFS 
programme in Kenya included 19 different crops and this was too high a number to 
ensure the technical quality of the curriculum, and a smaller number of crops would 
likely have improved the quality of the intervention. In Zimbabwe farmers felt that 
the introduction of organic production to the FFS curriculum was overwhelming. 
Adding this additional component after farmers had practised FFS for a few seasons 
may have been better (Hofisi, 2003) . 

Both studies from Indonesia found inadequate coverage of pesticides in the 
curriculum (Van de Fliert, 1993; Winarto, 2004), including a failure to explain the 
theory underlying I PM and pesticides appropriately, making the curriculum less 
convincing to farmers (Winarto, 2004). The authors note that this omission may 
have led to inappropriate pesticide use. For example, farmers to a certain extent 
replaced pesticide- spraying with granular applications, or started applying lower 
doses of pesticides which can accelerate the development of pesticide- resistant pest 
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varieties (Van de Fliert, 1993) . The authors suggest a fear of appearing to approve of 
pesticide use may be the reason for the failure to indude this as a topic in the 
curriculum. 

Complexity of FFS content and coverage 

Nine of the studies included findings pertaining to the complexity, observability and 
relative advantage of FFS knowledge and practices (David, 2007; Dolly, 2009; 
Gottret & Cordoba, 2004; Hofisi, 2003; Machacha, 2008; Pahs, 2002; Simpson, 
1997; Van Der Wide, 2004; Winarto, 2004). 

Studies from Honduras and Indonesia found that the complexity of IPM made it 
difficult for farmers to implement all the IPM practices on their crops, as did 
insuffitient time and financial resources (Gottret & Cordoba, 2004; Van de Fliert, 
1993; Winarto, 2004). Some of the analytical tools used were too complex and 
percdved as impractical in relation to the time, energy and resources they required 
(Winarto, 2004). For instance, the use of forms to record fidd sampling with 
formulas to calculate percentages for damages and prevalence of insects was found 
to be of little practical use for the farmers who abandoned this in favour of simply 
recording what they observed in their fields (Van de Fliert, 1993) . 

Assessing FFS in Cameroon, David (2007) found that practical knowledge was 
more easily understood than theoretical concepts. When asked about what 
they learned in FFS farmers focused on practical aspects such as spedfic 
management practices and hardly ever mentioned prindples and concepts of agro- 
ecosystem analysis. 

All of these studies suggest that due to the complexity of the IPM curriculum 
observability is important for farmers to trust the messages and develop 
analytical skills (Dolly, 2009; Hofisi, 2oo3; Machacha, 2008; Palis, 2002; 
Simpson, 1997; Van DerWiele, 2004; Winarto, 2004). The observability of IPM 
practices and their relative advantage compared with conventional methods was 
found to be important in enhandng farmers' confidence and belief in the truth of 
IPM messages. 

For instance, in Indonesia demonstration plots allowed farmers to study IPM 
empirically, and in doing so appeared to be an important part of farmers' knowledge 
formation (Winarto, 2004). Through experimentation in the demonstration plot the 
farmers were able to observe and understand the concept of benefidal and harmful 
insects, fadlitating IPM practice (Palis, 2002) . While farmers feared that insects 
would spread from neighbouring fields, experimentation and observation changed 
farmers' beliefs about this. The farmers also observed that their yields remained the 
same even if they did not spray, and the author suggests that this explains why 
farmers continued to practise IPM. 

In some cases farmers had to observe the benefits of techniques in the field of other 
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fanners before implementing them in their own fields (Hofisi, 2003; Van Der Wiele, 
2004) . In the case of Liberia people were particularly risk averse and vulnerable, 
and observing the relative advantage of the practices promoted through FFS 
appeared to be particularly important (Van Der Wiele, 2004) . Farmers were initially 
reluctant to experiment with vegetable crops as it was perceived as risky, but were 
starting to consider diversifying their crops when noticing the relative advantage of 
vegetables. 

Failure to demonstrate observable benefits can also act as a barrier to adoption. In 
Cambodia the facilitators were unable to demonstrate that farmers' crops could be 
improved through knowledge and techniques, rather than inputs (Simpson, 1997). 
And in Trinidad and Tobago the FFS did not generate results which were sufficient 
to convince farmers of the relative advantage of IPM (Dolly, 2009). 


Service delivery quality 

The quality of the implementation of FFS is likely to influence effects of the 
intervention on farmers' outcomes. Five studies included evidence on the resources 
devoted to FFS and the quality of the training delivered to farmers (Hofisi, 2003; 
Machacha, 2008; Najjar, 2009; Pedersen et al„ 2008; Winarto, 2004). 

In Tanzania one study suggests that lack of adequate resources, including 
inputs and finances, was the main reason for failure (Pedersen et al., 

2008). The study found evidence of major implementation failures, including 
insufficient and incidental provision of inputs, and difficulties with logistics and 
dissemination. The number of implemented FFS fell short of those planned. The 
majority of financial resources did not reach the FFS, suggesting failed and delayed 
payments contributed to these problems. The discrepancies between budgeted and 
received amounts were substantial and a lack of knowledge of how much money the 
FFS should be receiving contributed to limited (financial) accountability to 
participants. 

Another study found that lack of funding and inappropriate FFS sites were 
limitations of the FFS programme in Kenya (Najjar, 2009). It reports a lack of 
funding for irrigation, and inappropriate selection of FFS sites. The sites were 
remote, with limited irrigation and poor soil fertility, limiting the crops and 
practices which could be grown at the demonstration plots. In Indonesia the lack of 
adequate follow-up and support during pest outbreaks was noted as a 
constraint on farmers' willingness to continue practising IPM during outbreaks 
(Winarto, 2004). 

In Zimbabwe FFS implementation was adapted to suit the needs of 
participating farmers (Hofisi, 2003). The training schedule was adjusted to 
better suit farmers' work schedules and the size of FFS groups was reduced to 
improve farmers' learning opportunities. In Kenya farmers felt that the regular 
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location and time of FFS meetings was an advantage and the time when 
meetings took place was also appropriate for parents as the FFS group met when 
children were in school (Machacha, 2008). 


Policy context and institutional set-up 

Six out of twenty studies discuss the role of policy context and institutional set-up in 
influencing the success of FFS (Isubikalu, 2007; Mandni et al., 2007; Pedersen et 
al., 2008; Simpson, 1997; Van de Fliert, 1993; Winarto, 2004). 

Four studies suggest that the institutional legacy of existing extension 
systems influenced the implementation of FFS in Uganda, India and 
Indonesia (Isubikalu 2007; Mandni et al., 2007; Van de Fliert, 1993; Winarto, 

2004). In Uganda the institutional structures assodated with earlier top-down 
agricultural extension systems, such as T&V remained and contradicted the bottom- 
up, partidpatoiy approach of FFS (Isubikalu, 2007). 

Similarly, while some supportive polities were in place in Indonesia, such as the 
banning of pestiddes and removal of subsidies, 77 conflicting mechanisms assodated 
with the conventional extension system continued to contradict I PM adoption (Van 
de Fliert, 1993; Winarto, 2004). The government- run rice intensification 
programmes, induding subsidised input schemes and trickle-down messages, 
remained (Winarto, 2004, pp. 337- 338). Under the subsidised credit scheme, 
farmers were offered a "preset technology package" - a forced selling scheme of 
inputs that did not allow farmers to choose the induded inputs or refuse unwanted 
items such as pestiddes and foliar fertilisers, despite government regulation to the 
contrary (Van de Fliert, 1993, p. 225; Winarto, 2004). 

Moreover, two studies suggest that the pesticide industry maintained close 
links with the extension system at the local level (Mandni et al., 2007; Van 
de Fliert, 1993). In both Indonesia and India the involvement of extension workers 
and local cooperatives in pestidde promotion may have hampered farmers' efforts to 
practise I PM. Extension workers acted as intermediaries in input distribution 
between the local cooperatives, salespeople and the farmers, receiving commission 
for input sales. Consequently some used their position to promote pestiddes and 
other inputs. 

Three studies suggest diverging institutional incentives and objectives 
influenced service delivery quality (Isubikalu, 2007; Pedersen et al., 2008; 
Simpson, 1997). In Uganda, the institutional and hierarchical structure of extension 


77 The Presidential Decree of 1986 (Inpres 3/ 1986) represented a "commitment to a national policy of 
Integrated Pest Management (IPM) to replace the method of pest control that depended on pesticides 
only (Oka 1991; Fox 1991)” (Winarto, 2004, p. 23). The decree induded a ban of broad- spectrum 
pestiddes, the removal of pestidde subsidies and gave high importance to the improvement in human 
resource development (through offidal instructions of both offidal agricultural offidals and farmers), 
and biological and cultural controls in pest management (Van de Fliert 1993; Winarto, 2004, p. 23). 
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services, where each involved party promotes the mandates of their institutions 
(international development community, national and local institutions), was found 
to be crowding out the beneficiaries' needs and interests (Isubikalu, 2007) . 
Similarly, a study from Cambodia suggests that the national government was 
"disconnected from the IPM-FFS initiative, acting only as a 'rubber stamp' for 
international aid organisation decisions" (Simpson, 1997, pp. 136- 137). 

In Tanzania, the spearheading of FFS from a regional level was identified as a 
barrier resulting in insufficient extension visits and lack of hands- on 
implementation (Pedersen et al., 2008). Two studies further note the potential role 
of decentralisation, institutional set-up and funding structure (Isubikalu, 2007; 
Pedersen et al., 2008) . For instance, according to the study in Uganda (Isubikalu, 
2007), the system was set up to prioritise implementation of projects whose 
objectives were the uptake of new or improved technologies. Decisions about which 
proj ects to accept or rej ect were made by senior civil servants who had little or no 
practical experience in the field and lacked understanding of the problems and 
needs of farmers. The involvement of a multitude of institutions resulted in each 
impacting on the design, content and implementation of Ugandan FFS. 


Availability of inputs, labour and markets 

Five studies highlight issues related to availability of inputs, labour and markets 
(DANIDA, 2011; Machacha, 2008; Mandni et al., 2007; Najjar, 2009; Van Der 
Wiele, 2004). Four studies from India, Bangladesh, Liberia and Kenya note the 
non-availability of inputs as a barrier to uptake of FFS practices (Mandni et 
al., 2007; DANIDA, 2011; Van Der Wiele, 2004; Machacha, 2008). The studies 
highlight factors such as lack of access to capital (DANIDA, 2011; Machacha, 2008), 
unavailability of inputs in the market (Mandni et al., 2007) and bad experiences 
with service providers cheating the farmers on the quality and quantity of 
agricultural products (DANIDA, 2011). In Kenya farmers noted that they pooled 
attention and inputs for the group plots, but that the necessary supplies and labour 
was often lacking on individual farms (Machacha, 2008, p. 64). A different study 
from Kenya found that both men and women emphasised access to markets and 
funding as essential factors for enhandngthe FFS programme (Najjar, 2009). 


Community and context 

Contextual fadors are potentially important in moderating FFS effectiveness. Seven 
studies discuss the role of the community and contextual factors as enablers of and 
barriers to FFS effectiveness (Dolly, 2009; Machacha, 2008; Najjar, 2009; Palis, 
2002; Simpson, 1997; Van Der Wiele, 2004; Winarto, 2004). 

Four studies from Trinidad and Tobago, Nicaragua, the Philippines and Cambodia 
included themes related to the role of existing social capital in influencing 
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FFS effectiveness (Dolly, 2009; Gottret & Cordoba, 2004; Palis, 2002; Simpson, 
1997). In Trinidad and Tobago, Nicaragua and the Philippines collective action 
through existing fanner groups may have facilitated group learning and action 
(Dolly, 2009; Gottret & Cordoba, 2004; Palis, 2002). For instance, in the 
Philippines, FFS participants were members of a cooperative and the "camaraderie 
existing among the members of the cooperative gave the participants a collective will 
to learn and succeed in the training" (Palis, 2002, p. 78). The study suggests that 
this gave farmers the courage to implement IPM, including not spraying their fields, 
which farmers were particularly anxious about. 

In the Philippines social capital was found to facilitate sustained IPM diffusion and 
adoption (Palis, 2002) . FFS villages were characterised by high levels of social 
capital, with conversations and informal discussions taking place in the 
neighbourhood, the market, the pavilion and the farm (Palis, 2002). Social capital 
was high among kin networks, household neighbourhoods, farm neighbourhoods 
and farmers' associations. The chairman of the farmers' association was a trusted 
opinion leader and played an important role in convincing the farmers not to spray 
by encouraging them to unite to fight the pests and trust the IPM practices 
promoted in the FFS. 

On the other hand, Simpson ( 1997) suggests that in Cambodia the low level of social 
capital influenced how farmers perceived some of the FFS practices, including 
collective action. He found that there was little sense of community in the FFS 
villages, with the "communities" not being more than geographic entities, 
characterised by class divisions and individualistic practices. 

Similarly, farmers in Indonesia found that practising IPM was challenging 
when not reinforced by the rest of the community and FFS farmers found it 
difficult to practise new strategies when the rest of the community continued 
applying "old" practices (Winarto, 2004). On the other hand, when the cultural rules 
about the appropriate pest- control strategies for a pest were still "in the making", as 
was the case during a pest outbreak of the white rice stem borer, the FFS- IPM 
approach seemed to prevail, facilitated by the ability of FFS-IPM farmers to reach a 
consensus about the most appropriate strategy for its control (p. 357) . However, the 
farmers still had to face discouragement and disbelief from their untrained peers, 
especially at the early stages of introducing the new ideas. To counteract this 
pressure, the trained farmers sought to legitimise their new identities as schooled 
farmers in the community. 

In Liberia a context of vulnerability affected the decision-making of FFS 
households (Van Der Wiele, 2004) . Due to chronic conflict and instability people's 
planning horizons were relatively short term, with overuse of resources and under- 
investment in long-term projects characterising farming practices. Thus, practices 
reguiring less time and resources investments were more likely to be adopted. 
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Diffusion to non-participant farmers 


The extent to which the practices promoted in FFS diffuse to non-FFS fanners is a 
key issue in the debate about FFS effectiveness and cost- effectiveness. Eleven 
studies included themes relevant to the diffusion of FFS knowledge and practices to 
non- participants (David, 2007; Gotret &Cordoba, 2004; Hiller, 2009; Karanja- 
Lumumba et al., 2007; Machacha, 2008; Mandni etal., 2007; Palis, 2002; Rola& 
Baril, 1997; Simpson, 1997; VandeFliert, 1993; Winarto, 2004). 

As in the case of knowledge acquisition and adoption by FFS participants, 
complexity and observability appear to be factors affecting diffusion of knowledge 
and practices to the broader community. Four studies highlight the complexity 
and the experiential nature of FFS learning as a barrier to diffusion 
(David, 2007; Mandni et al., 2007; Van de Fliert, 1993; Winarto, 2004). They note 
that despite high awareness of IPM by non-partidpants (Van de Fliert, 1993; 
Winarto, 2004), the skills and practices are complex and their experiential nature 
makes them difficult to convey via verbal communication (Mandni et al., 2007; Van 
de Fliert, 1993; Winarto, 2004). 

Thus Winarto concludes, "even though knowledge transmission to the lay farmers 
did occur throughout the course of pest outbreaks, the received ideas were not 
always in the same form or were not comprehensively disseminated" (Winarto, 

2004, p. 356) . FFS partidpants also felt that their understanding of the meaning and 
interpretation of developments in the field diverged from those of the non- 
partidpants, which also undermined diffusion. In another study non-partidpants 
reported not feeling confident to implement the new practices they had heard about 
from their FFS neighbours (Mandni et al., 2007). 

In two studies where diffusion was observed, the findings suggest concrete 
practices were more likely to diffuse than theoretical concepts and 
principles (David, 2007; Hiller et al., 2009), with relatively easy practices, such as 
pruning and weeding techniques, being more easily disseminated. 

Seven studies highlight the importance of observability for convincing non- 
FFS farmers to adopt FFS practices (David, 2007; Gotret & Cordoba, 2004; Pahs, 
2002; Machacha, 2008; Simpson, 1997; Winarto, 2004; Van de Fliert, 1993). For 
instance, Winarto notes that observing the pests in their own fields and observing 
the practices of FFS partidpants were important in building non-partidpants' 
understanding of IPM practices. Several studies suggest that observing the 
successful harvests of FFS farmers triggered interest and requests for advice from 
non-partidpants (David, 2007; Machacha, 2008; Pahs, 2002). On the other hand, in 
Cambodia the results observable in IPM farmers' plots were less convindng and 
hence, non- IPM farmers were not convinced of the benefits of IPM ( Simpson, 1997) . 

In both Honduras and Kenya non-FFS farmers perceived FFS practices as 

having a relative advantage compared with existing practices, fadhtating 
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interest in IPM (Gotret & Cordoba, 2004; Hiller et al., 2009). The study from 
Honduras suggests that the perceived economic advantage of FFS was an important 
factor in convincing non- FFS farmers to adopt IPM. In Kenya non- FFS farmers were 
most interested in information about practices aimed at increasing productivity. 

As in the case of knowledge acquisition and adoption by IPM participants, social and 
contextual factors may also be important for diffusion. Five studies included 
findings suggesting existing levels of social capital play a role in diffusion of 
FFS knowledge and practices (David, 2007; Gotret &Cordoba, 2004; Palis, 
2002; Rola &Baril, 1997; Simpson, 1997). In the Philippines social capital, and in 
particular kinship ties, facilitated diffusion of IPM (Palis, 2002; Rola and Baril, 

1997) . Farmer sharing of IPM concepts with non-IPM farmers was found to be 
directed more towards kinship - this is attributed to the Filipino normative 
expectations and obligations among members in a kin group, especially siblings and 
members of the nuclear household. The author recommends using social capital in 
the form of kinship ties and farm location to enhance the efficiency of farmer- to- 
farmer diffusion of IPM (Palis, 2002). 

On the other hand, in both Cambodia and Cameroon, low levels of social capital and 
limited reach of social networks were found to act as barriers to diffusion. In 
Cameroon FFS participants explained their lack of sharing of knowledge from FFS 
with others by the fact that all their friends were also FFS participants (David, 

2007). In Cambodia low levels of social capital and cohesion limited communication 
within the community, preventing diffusion of IPM knowledge (Simpson, 1997). 

One study highlights that socioeconomic differences between FFS 
participants and non-participants impeded diffusion (Van de Fliert, 1993). 
The Indonesian FFS was designed to encourage participants to train other group 
members and promote communication with the rest of the community, for example 
through field days and folk theatre. However, the author found that the non- 
representative composition of the FFS groups impeded interaction between 
participants and non- participants. FFS participants communicated to a "selective 
audience in the villages" and made no deliberate efforts to train other members of 
the community in IPM principles. At the same time, the status difference between 
group members and the untrained farmers seems to also have contributed to the 
disinterest of the community in the organised field days (Van de Fliert, 1993) . 

Another study from Indonesia highlights the role of opinion leaders in 
dissemination of knowledge about IPM practices. A few inquisitive farmers 
played a prominent role in the ongoing process of knowledge formulation and 
transmission. These farmers progressively established their position within the 
community as "experts", "farmer professors" and "consultants" (Winarto, 2004, p. 
351) . However, the study also notes the difficulties of this process in the face of 
unyielding perspectives. The author notes that "without the provision of consistent 
policies, assistance, and rewards from fellow farmers and government officials, these 
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fanners had difficulty maintaining their spirits and the efforts to learn" (p. 351) . The 
importance of an organised approach to diffusion is further highlighted by the 
experience from Kenya, where FFS networks 78 actively encouraged diffusion 
of adoption. The FFS networks supported information- sharing among farmers, 
organised field days, exchange visits and tours; and set up demonstration fields 
(Karanja-Lumumbaetal., 2007). 


Sustainability 

Sustainability of FFS practices in farmer communities is an important issue for cost- 
effectiveness of FFS interventions. Eleven out of twenty studies discuss factors 
affecting sustainability of FFS (DANIDA, 2011; David, 2007; Dolly, 2009; Friis- 
Hansen, 2008; Gottret & Cordoba, 2004; Karanja-Lumumbaetal., 2007; 

Machacha, 2008; Simpson, 1997; Van deFliert, 1993; Van DerWiele, 2004; 
Winarto, 2004). 

Seven studies suggest that ongoing support and/or follow-up are important 
for sustainability of FFS practices (DAMDA, 2011; David, 2007; Dolly, 2009; 
Gottret & Cordoba, 2004; Simpson, 1997; Van DerWiele, 2004; Winarto, 2004). 
Four studies note lack of ongoing support and follow-up as an important barrier to 
the sustainability of the FFS approach (Dolly, 2009; Gottret & Cordoba, 2004; 
Simpson, 1997; Winarto, 2004). In Indonesia farmers also reported a lack of 
"consistent support to back up their struggles in creating and maintaining the new 
schemas of interpretations and practices" in the face of recurrent pest and disease 
outbreaks and continuing recommendations to use pesticides (Winarto, 2004, p. 
363) . In Trinidad and Tobago, farmer associations felt that there was not enough 
technical assistance from researchers to support farmers in continuing development 
of local practices (Dolly, 2009). A study from Honduras highlights insufficient 
support from the implementing agency to the local partner organisations, their 
ability to support farmers being limited by lack of time and access to technical 
backstopping (Gottret & Cordoba, 2004). Similarly, another study mentioning the 
absence of follow-up activities notes that there was a lack of clarity in 
communication regarding the responsibility for follow-up activities between the 
implementing agency and the local NGO ( Simpson, 1997) . 

Four studies highlight the role of support and follow-up on the 
establishment and sustainability of FFS-related activities (DAMDA, 2011; 
Karanja-Lumumbaetal., 2007; Machacha, 2008; Van DerWiele, 2004). In Liberia, 
the implementing agency hoped that FFS groups would develop into community- 


78 The FFS networks arose independently of the FAO implemented East African Sub- Regional Pilot 
Project for Farmer Field Schools, with the aim to "sustain Farmer Field Schools, link Farmer Field 
Schools to input and output markets, link farmers to information sources and facilitate information 
flow, form a forum for Farmer Field Schools for information exchange and experience sharing in 
relation to farming and promote the FFS concept as an extension methodology in addressing emerging 
issues especially in agricultural development" ( Karanj a- Lumumba et al„ 2007, p. 1,347). 
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based organisations that would continue meeting and working together. The study 
relates the positive results of the most successful of these groups to the involvement 
of implementation staff, noting that it was the only group that received follow-up 
and support from the agency (Van Der Wiele, 2004) . In Bangladesh, FFS fanners 
were encouraged to establish fanner dubs, which continued to be supported by the 
implementing agency. The authors suggest that additional sessions on club 
formation were "an asset to the establishment of sustainable and effective groups 
and an improvement of the practices used" (DAMDA, 2011, p. 36). Two studies from 
Kenya found that there was no support for FFS- related follow-up activities and the 
sustainability of group activities depended on the willingness of offidals to serve on 
a voluntary basis, as well as the capadties of the different FFS groups (Karanja- 
Lumumba et al., 2007; Machacha, 2008). 

The motivation of FFS partidpants is likely to be an important factor for the 
sustainability of FFS practices after completion of the training. However, only one 
study reports on this issue. The study found that reimbursing participants for 
FFS attendance may have undermined sustainability of the FFS groups 
(Simpson, 1997), noting that FFS students did not see themselves continuing to 
learn and share information following graduation. 

Four studies highlight group- related factors affecting sustainability (DAMDA, 2011; 
David, 2007; Machacha, 2008; Van Der Wide, 2004). One study mentions 
consistent membership participation being important for sustainability 
(Machacha, 2008) . A study from Cameroon found group support and validation 
important in building up confidence in FFS practices of FFS graduates 
(David, 2007) and three studies found that strong collective goals and activities 
were important for the sustainability of the groups after the end of FFS 
training (Van de Fliert, 1993; Van Der Wiele, 2004) . The leader of a successful FFS 
group turned community organisation attributed the group's ability to start and 
successfully complete projects to a common understanding/ mindset among 
partidpants, as well as respect among the members, regularity of meetings and a 
high interest by group members (Van Der Wiele, 2004) . A study from Indonesia 
found that in the absence of a common pest problem, or when the group members' 
and leaders' interests diverged, farmer would stop engaging in collective follow-up 
activities (Van de Fliert, 1993). In another study from Bangladesh, members of 
farmer assodations and farmer dubs noted the importance of leadership in the 
sustainability of farmer clubs established from FFS groups (DAMDA, 2011). 


Perceived outcomes: gender and empowerment 

Only four studies included in the effectiveness synthesis collected quantitative data 
on some measure of empowerment. The analysis suggests a benefidal effed on the 
probability of the farmers positively answering questions on self-esteem, induding 
feeling capable of solving problems in the field, feeling comfortable in giving an 
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opinion, and participating in the community. However, empowerment is a multi- 
dimensional concept that is inherently difficult to measure. The impact of an 
intervention on empowerment is context- and time- specific, and difficult to capture 
directly. The use of a single index is insufficient and combined indices can often hide 
the differential impact of an intervention on the various dimensions of 
empowerment (Narayan, 2005). To complement the findings of the effectiveness 
studies this section reports on findings from qualitative studies regarding farmers' 
perceptions about the role of FFS in gender relations and empowerment. 

Eleven out of twenty studies discuss the role of FFS in gender relations and farmer 
empowerment (DANIDA, 2011; Dolly, 2009; Friis- Hansen, 2008; Friis- Hansen et 
al., 2012; Hofisi, 2003; Machacha, 2008; Mandni etal., 2007; Najjar, 2009; 
Simpson, 1997; VanDerWiele, 2004; Winarto, 2004). 

Empowerment 

All ten studies discussing empowerment of participants suggest that participation in 
FFS was an empowering experience for the participating farmers (Dolly, 2009; Friis- 
Hansen, 2008; Friis- Hansen et al., 2012; Hofisi, 2003; Machacha, 2008; Mandni et 
al., 2007; Najjar, 2009; Simpson, 1997; VanDerWiele, 2004; Winarto, 2004). Some 
studies also found that these benefits extended to the non-partidpants in the 
intervention communities. 

Nine studies suggest that FFS led to improved confidence, decision-making 
skills and enhanced agency. Fanners felt that FFS increased their self- 
confidence (Dolly, 2009; Friis- Hansen, 2008; Friis- Hansen etal., 2012; Hofisi, 

2003; Machacha, 2008; Mandni et al., 2007; Simpson, 1997; Van Der Wide, 2004; 
Winarto, 2004), dedsion- making skills in agriculture (Hofisi, 2003; Mandni et al., 
2007) and improved their ability to articulate their demands to service providers 
and engage with stakeholders (Dolly, 2009; Friis- Hansen, 2008). 

Winarto suggests that these increases in self-confidence in Indonesia were gained 
during farmers' efforts in resolving the various problems they encountered during 
and following their FFS- IPM training (Winarto, 2004). Similarly, a study from 
Zimbabwe suggests that farmer- driven experimentation has taught farmers to 
recognise their ability to identify and solve problems, improving self-confidence 
(Hofisi, 2003) . The farmers fdt confident enough to take on leadership roles, 
induding facilitation of FFS sessions (Friis- Hansen, 2008; Friis- Hansen et al., 2012; 
Winarto, 2004). However, in one study, the authors note doubts about the extent to 
which increases in self-confidence would translate into future activities, noting that 
farmers did not feel confident enough to carry out their own experiments (Simpson, 
1997). 

In two studies, FFS partidpants reported perceiving stronger social ties, 
improved collaboration and collective action (Mandni et al., 2007; Najjar, 
2009). In an Indian FFS, the gains in sodal capital also extended to the non- 
partidpating neighbours in the intervention communities through the formation of 
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dubs that conducted on-site experimentation and sodal activities designed to 
support the poor and vulnerable members of the community (Mandni et al., 2007). 

Four studies also mention that FFS graduates acquired enhanced status within 
the community (Friis- Hansen, 2008; Friis- Hansen et al., 2012; Machacha, 2008; 
Winarto, 2004) . FFS farmers felt that they were treated with more respect 
(Machacha, 2008), and adopted leadership roles, becoming informal leaders in 
Kenya (Friis- Hansen et al., 2012), fanner group leaders in Uganda (Friis- Hansen, 
2008) and IPM experts or "farmer professors" in Indonesia (Winarto, 2004). In the 
last case, the authors note that "the sodal positions or categories of the famers have 
become less significant than the reliable and valid information they could provide", 
suggesting that FFS knowledge allowed farmers to break out of their traditional 
community roles and relationships (Winarto, 2004, p. 351). 

Gender 

Six out of twenty studies directly discuss the part of FFS in gender roles and 
relationships and women's empowerment spedfically (DAMDA, 2011; Dolly, 2009; 
Friis- Hansen et al., 2012; Machacha, 2008; Mandni et al., 2007; Najjar, 2009). All 
of these studies suggest that FFS contributed to women's personal 
empowerment (DAMDA, 2011; Dolly, 2009; Friis- Hansen et al., 2012; Machacha, 
2008; Mandni et al., 2007; Najjar, 2009). The studies present subjective reports of 
women perceiving an increased sense of confidence (DAMDA, 2011; Dolly, 2009; 
Friis- Hansen et al., 2012). In India, women perceived that their human and sodal 
capital had increased following FFS partidpation, and that FFS allowed them to gain 
recognition for their skills and abilities. In Kenya women from one FFS reported 
that they were "beginning to recognize themselves as a viable source of knowledge, 
particularly for issues outside the maintenance of the household (e.g. farming) " 
(Friis- Hansen et al., 2012, p. 52). In Trinidad and Tobago, women perceived that 
FFS had increased their self-confidence to share their ideas on matters where they 
would previously not have done so (Dolly, 2009), and in Bangladesh, women felt 
more confident in public speaking. They also reported that their status and role in 
the household and the community had improved (DAMDA, 20 11) . 

One study from Kenya found that FFS have established a favourable group 
composition and atmosphere that enabled breaching traditional 
community roles and relationships (Friis- Hansen et al., 2012). The study 
described an FFS that established a dynamic and positive atmosphere in the groups 
and a structure.that dictated that all participants should be equally involved in all 
activities. This, according to the authors, led to the creation of a "safe space" where 
both women and men could take on roles not traditionally accepted outside the 
groups, and collaborate on carrying out practices that were normally gendered 
(Friis- Hansen et al., 2012, pp. 49- 53). The FFS participants also reported that the 
participation in FFS had improved gender relations within groups and at 
household level, including men's regard and opinion of women. The changed 
attitude to women appeared to be a key factor in changing household relations. 
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inducting increased collaboration and joint dedsion- making. Households reported a 
reduction in quarrels, and improved incomes thanks to the increased economic 
activities of women, inducting in commercial agriculture which was traditionally 
considered a taboo (Friis- Hansen et al„ 2012). 

However, two studies suggest that women did not always acquire greater 
economic and other decision-making powers (DANIDA, 2011; Najjar, 2009). 
A study from Bangladesh found that, while women were more likdy to be consulted, 
men still had the final say in "big dedsion- making" on issues such as larger 
agricultural investments and land ownership, child marriage, child labour, 
polygamy, male employment and migration. Women's control over income has in 
the maj ority of cases not improved, with men retaining dedsion- making power on 
women's spending. This was the case even when women were allowed to take out 
loans from farmer groups (DANIDA, 20 11) . 

A study from India suggests that "the increased household cash flow - perceived by 
both men and women IPM-FFS partidpants - translated into the purchase of new 
physical assets mostly for men" (Mandni et al., 2007, p. 108). In Bangladesh and 
Kenya, women and minorities remained poorly represented in the leadership and 
dedsion-making of local organisations (DANIDA, 2011; Najjar, 2009). The authors 
of the study from Bangladesh suggest that the limited impact on female 
empowerment was largely due to a failure to fully address intra- household relations 
and sodo- cultural / gender issues, noting that engagement with gender and sodo- 
cultural issues were considered "add-ons" rather than integral parts of the FFS 
approach (DANIDA, 2011). 

Three other studies similarly found that traditional gender roles remained 
within the groups (Dolly, 2009; Machacha, 2008; Najjar, 2009). Two studies 
found that men's views dominated in the FFS group fora and proceedings (Dolly, 
2009; Machacha, 2008). A study from Kenya found gender preferences for group 
composition resulting in gender- segregated groups. Men traditionally preferred 
single- sex groups. Women preferred mixed groups so that men could take on some 
traditionally male tasks. Within mixed groups, men adopted production- oriented 
male roles, while women would persevere in subsistence- oriented and reproductive 
roles. Where groups were mixed, FFS did not address gender balances sufficiently. 
Men would adopt leadership positions even where FFS groups were composed 
predominantly of women participants. Cultural roles (which for instance prevented 
travel) and household duties were reported as barriers to women taking on more 
active roles (Najjar, 2009). 

The studies from Kenya seem to have contradictory findings with regards to the 
success of FFS in breaking down traditional gender roles. Unfortunately, since the 
Friis- Hansen studies (2008, Friis- Hansen et al., 2012) do not provide any 
information about the FFS project under study, it is not possible to evaluate whether 
this difference is due to differences in project design or implementation, or whether 
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in fact, the studies refer to the same project but reach contradictory conclusions. 
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